[Test Proxy] Add test recording fixture

[Laia Patiño (laiapat)]

Description

This takes an alternative approach to recording and playing back tests: using a fixture instead of a wrapping decorator.

All SDK Contribution checklist:

• The pull request does not introduce [breaking changes]
• CHANGELOG is updated for new features, bug fixes or other significant changes.
• I have read the contribution guidelines.

General Guidelines and Best Practices

• Title of the pull request is clear and informative.
• There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

• Pull request includes test coverage for the included changes.

[Azure SDK Bot (azure-sdk)]

API change check

APIView has identified API level changes in this PR and created following API reviews.

azure-data-tables
azure-keyvault-keys

[Scott Beddall (scbedd)]

This is looking really good McCoy. We are missing some integration checking, and that's pretty much it right?

[Laia Patiño (laiapat)]

Yep, we're almost there! I just need to add async functionality, make a few tweaks to the variable recorder for easier use, and work out any integration and invocation kinks (like applying the fixture to tests from a class or session level). I'll mark this ready for review and provide an update when it's there 🙂

[Scott Beddall (scbedd)]

In terms of integrating this PR with the rest of the repo. Perhaps we maintain the existing recorded_by_proxy so that we can merge this and make forward progress while migrating previous implementations.

[Laia Patiño (laiapat)]

In terms of integrating this PR with the rest of the repo. Perhaps we maintain the existing recorded_by_proxy so that we can merge this and make forward progress while migrating previous implementations.

I completely agree. That raises a point about the main blocker for marking this ready for review: the variables fixture isn't compatible with parameterized tests. So, for example, most Key Vault tests can't use the variable_recorder fixture.

That isn't an immediate issue since: 1. KV tests don't use variables today, and 2. the recorded_by_proxy decorator will still be available as an alternative. But it's something I'd like to figure out, though I haven't been able to after a few days of trying.

Since the decorator approach to recording will still be supported, should we try to get this merged sooner rather than later, once the small packaging issue is resolved? The ML team is hoping to migrate their tests soon, and they would need this recording fixture to make that happen. Scott Beddall (@scbedd) Laurent Mazuel (@lmazuel)

[Laurent Mazuel (lmazuel)]

Talked to @mccoyp offline, thumbs up. ;)

[Laia Patiño (laiapat)]

/azp run python - keyvault - ci

[azure-pipelines]

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

[Laia Patiño (laiapat)]

Build verifying that conftest.py gets loaded correctly for servicebus (the only other service that has a dev_requirements.txt excluding azure-sdk-tools -- specifically in a perf-specific file): https://dev.azure.com/azure-sdk/public/_build/results?buildId=1722981&view=results

[Scott Beddall (scbedd)]

This is great McCoy. Maybe Anna Tisch (@annatisch) will pick something out that I'm not seeing, but this looks super solid.

[Anna Tisch (annatisch)]

LGTM :)