implement --only for emulators:export#10057
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
The pull request implements the --only flag for the emulators:export command, allowing users to selectively export data for specific emulators. This involved modifying the CLI controller, emulator hub, and export logic to filter emulators based on the provided targets. New end-to-end tests were added to validate this functionality, which also included cleanup for admin SDK instances in existing tests. A review comment highlighted significant code duplication within these new test cases, suggesting refactoring into a shared helper function to improve maintainability.
| it("should export all data when `--only` flag isn't used `emulators:export`", async function (this) { | ||
| this.timeout(2 * TEST_SETUP_TIMEOUT); | ||
| await new Promise((resolve) => setTimeout(resolve, 2000)); | ||
|
|
||
| // Start up emulator suite | ||
| const emulatorsCLI = new CLIProcess("1", __dirname); | ||
| await emulatorsCLI.start( | ||
| "emulators:start", | ||
| FIREBASE_PROJECT, | ||
| ["--only", "storage,auth"], | ||
| logIncludes(ALL_EMULATORS_STARTED_LOG), | ||
| ); | ||
|
|
||
| const credPath = path.join(__dirname, "service-account-key.json"); | ||
| const credential = fs.existsSync(credPath) | ||
| ? admin.credential.cert(credPath) | ||
| : admin.credential.applicationDefault(); | ||
|
|
||
| const config = readConfig(); | ||
| const storagePort = config.emulators!.storage.port; | ||
| process.env.STORAGE_EMULATOR_HOST = `http://${await localhost()}:${storagePort}`; | ||
|
|
||
| // Write some data to export | ||
| const aApp = admin.initializeApp( | ||
| { | ||
| projectId: FIREBASE_PROJECT, | ||
| storageBucket: "bucket-a", | ||
| credential, | ||
| }, | ||
| "storage-export-a", | ||
| ); | ||
| const bApp = admin.initializeApp( | ||
| { | ||
| projectId: FIREBASE_PROJECT, | ||
| storageBucket: "bucket-b", | ||
| credential, | ||
| }, | ||
| "storage-export-b", | ||
| ); | ||
|
|
||
| // Write data to two buckets | ||
| await aApp.storage().bucket().file("a/b.txt").save("a/b hello, world!"); | ||
| await aApp.storage().bucket().file("c/d.txt").save("c/d hello, world!"); | ||
| await bApp.storage().bucket().file("e/f.txt").save("e/f hello, world!"); | ||
| await bApp.storage().bucket().file("g/h.txt").save("g/h hello, world!"); | ||
|
|
||
| // Create some accounts to export: | ||
| const authPort = config.emulators!.auth.port; | ||
| process.env.FIREBASE_AUTH_EMULATOR_HOST = `${await localhost()}:${authPort}`; | ||
| const cApp = admin.initializeApp( | ||
| { | ||
| projectId: FIREBASE_PROJECT, | ||
| credential: ADMIN_CREDENTIAL, | ||
| }, | ||
| "auth-export", | ||
| ); | ||
| await cApp.auth().createUser({ uid: "123", email: "foo@example.com", password: "testing" }); | ||
| await cApp.auth().createUser({ uid: "456", email: "bar@example.com", emailVerified: true }); | ||
|
|
||
| // Ask for export | ||
| const exportCLI = new CLIProcess("2", __dirname); | ||
| const exportPath = fs.mkdtempSync(path.join(os.tmpdir(), "emulator-data")); | ||
| await exportCLI.start( | ||
| "emulators:export", | ||
| FIREBASE_PROJECT, | ||
| [exportPath], | ||
| logIncludes("Export complete"), | ||
| ); | ||
| await exportCLI.stop(); | ||
|
|
||
| // Check that the right export files are created | ||
| const storageExportPath = path.join(exportPath, "storage_export"); | ||
| const storageExportFiles = fs.readdirSync(storageExportPath).sort(); | ||
| expect(storageExportFiles).to.eql(["blobs", "buckets.json", "metadata"]); | ||
|
|
||
| // Stop the suite | ||
| await emulatorsCLI.stop(); | ||
|
|
||
| // Attempt to import | ||
| const importCLI = new CLIProcess("3", __dirname); | ||
| await importCLI.start( | ||
| "emulators:start", | ||
| FIREBASE_PROJECT, | ||
| ["--only", "storage,auth", "--import", exportPath], | ||
| logIncludes(ALL_EMULATORS_STARTED_LOG), | ||
| ); | ||
|
|
||
| // List the files | ||
| const [aFiles] = await aApp.storage().bucket().getFiles({ | ||
| prefix: "a/", | ||
| }); | ||
| const aFileNames = aFiles.map((f) => f.name).sort(); | ||
| expect(aFileNames).to.eql(["a/b.txt"]); | ||
|
|
||
| const [bFiles] = await bApp.storage().bucket().getFiles({ | ||
| prefix: "e/", | ||
| }); | ||
| const bFileNames = bFiles.map((f) => f.name).sort(); | ||
| expect(bFileNames).to.eql(["e/f.txt"]); | ||
|
|
||
| const user1 = await cApp.auth().getUserByEmail("foo@example.com"); | ||
| expect(user1.passwordHash).to.match(/:password=testing$/); | ||
| const user2 = await cApp.auth().getUserByEmail("bar@example.com"); | ||
| expect(user2.emailVerified).to.be.true; | ||
|
|
||
| await importCLI.stop(); | ||
|
|
||
| // Clean up the admin sdk instances to prevent "Firebase app named <name> already exists." errors in later tests | ||
| await aApp.delete(); | ||
| await bApp.delete(); | ||
| await cApp.delete(); | ||
| }); | ||
|
|
||
| it("should export only storage data with `emulators:export --only storage`", async function (this) { | ||
| this.timeout(2 * TEST_SETUP_TIMEOUT); | ||
| await new Promise((resolve) => setTimeout(resolve, 2000)); | ||
|
|
||
| // Start up emulator suite | ||
| const emulatorsCLI = new CLIProcess("1", __dirname); | ||
| await emulatorsCLI.start( | ||
| "emulators:start", | ||
| FIREBASE_PROJECT, | ||
| ["--only", "storage,auth"], | ||
| logIncludes(ALL_EMULATORS_STARTED_LOG), | ||
| ); | ||
|
|
||
| const credPath = path.join(__dirname, "service-account-key.json"); | ||
| const credential = fs.existsSync(credPath) | ||
| ? admin.credential.cert(credPath) | ||
| : admin.credential.applicationDefault(); | ||
|
|
||
| const config = readConfig(); | ||
| const storagePort = config.emulators!.storage.port; | ||
| process.env.STORAGE_EMULATOR_HOST = `http://${await localhost()}:${storagePort}`; | ||
|
|
||
| // Write some data to export | ||
| const aApp = admin.initializeApp( | ||
| { | ||
| projectId: FIREBASE_PROJECT, | ||
| storageBucket: "bucket-a", | ||
| credential, | ||
| }, | ||
| "storage-export-a", | ||
| ); | ||
| const bApp = admin.initializeApp( | ||
| { | ||
| projectId: FIREBASE_PROJECT, | ||
| storageBucket: "bucket-b", | ||
| credential, | ||
| }, | ||
| "storage-export-b", | ||
| ); | ||
|
|
||
| // Write data to two buckets | ||
| await aApp.storage().bucket().file("a/b.txt").save("a/b hello, world!"); | ||
| await aApp.storage().bucket().file("c/d.txt").save("c/d hello, world!"); | ||
| await bApp.storage().bucket().file("e/f.txt").save("e/f hello, world!"); | ||
| await bApp.storage().bucket().file("g/h.txt").save("g/h hello, world!"); | ||
|
|
||
| // Create some accounts to export: | ||
| const authPort = config.emulators!.auth.port; | ||
| process.env.FIREBASE_AUTH_EMULATOR_HOST = `${await localhost()}:${authPort}`; | ||
| const cApp = admin.initializeApp( | ||
| { | ||
| projectId: FIREBASE_PROJECT, | ||
| credential: ADMIN_CREDENTIAL, | ||
| }, | ||
| "auth-export", | ||
| ); | ||
| await cApp.auth().createUser({ uid: "123", email: "foo@example.com", password: "testing" }); | ||
| await cApp.auth().createUser({ uid: "456", email: "bar@example.com", emailVerified: true }); | ||
|
|
||
| // Ask for export | ||
| const exportCLI = new CLIProcess("2", __dirname); | ||
| const exportPath = fs.mkdtempSync(path.join(os.tmpdir(), "emulator-data")); | ||
| await exportCLI.start( | ||
| "emulators:export", | ||
| FIREBASE_PROJECT, | ||
| [exportPath, "--only", "storage"], | ||
| logIncludes("Export complete"), | ||
| ); | ||
| await exportCLI.stop(); | ||
|
|
||
| // Check that the right export files are created | ||
| const storageExportPath = path.join(exportPath, "storage_export"); | ||
| const storageExportFiles = fs.readdirSync(storageExportPath).sort(); | ||
| expect(storageExportFiles).to.eql(["blobs", "buckets.json", "metadata"]); | ||
|
|
||
| // Stop the suite | ||
| await emulatorsCLI.stop(); | ||
|
|
||
| // Attempt to import | ||
| const importCLI = new CLIProcess("3", __dirname); | ||
| await importCLI.start( | ||
| "emulators:start", | ||
| FIREBASE_PROJECT, | ||
| ["--only", "storage,auth", "--import", exportPath], | ||
| logIncludes(ALL_EMULATORS_STARTED_LOG), | ||
| ); | ||
|
|
||
| // List the files | ||
| const [aFiles] = await aApp.storage().bucket().getFiles({ | ||
| prefix: "a/", | ||
| }); | ||
| const aFileNames = aFiles.map((f) => f.name).sort(); | ||
| expect(aFileNames).to.eql(["a/b.txt"]); | ||
|
|
||
| const [bFiles] = await bApp.storage().bucket().getFiles({ | ||
| prefix: "e/", | ||
| }); | ||
| const bFileNames = bFiles.map((f) => f.name).sort(); | ||
| expect(bFileNames).to.eql(["e/f.txt"]); | ||
|
|
||
| await expect(cApp.auth().getUserByEmail("foo@example.com")) | ||
| .to.eventually.be.rejectedWith(Error) | ||
| .and.have.property("code", "auth/user-not-found"); | ||
| await expect(cApp.auth().getUserByEmail("bar@example.com")) | ||
| .to.eventually.be.rejectedWith(Error) | ||
| .and.have.property("code", "auth/user-not-found"); | ||
|
|
||
| await importCLI.stop(); | ||
|
|
||
| // Clean up the admin sdk instances to prevent "Firebase app named <name> already exists." errors in later tests | ||
| await aApp.delete(); | ||
| await bApp.delete(); | ||
| await cApp.delete(); | ||
| }); |
There was a problem hiding this comment.
There's a significant amount of duplicated code between the two new test cases: should export all data when --only flag isn't used emulators:export and should export only storage data with emulators:export --only storage. The setup logic for starting emulators, initializing admin apps, and creating test data is nearly identical.
To improve test maintainability and reduce redundancy, consider refactoring this common setup into a shared helper function. This would make the tests cleaner and easier to manage in the future.
* implement --only for emulators:export * remove console logs * changelog entry * change to be nullable * add tests to make sure exporting with POST works * Update CHANGELOG.md --------- Co-authored-by: Joe Hanley <joehanley@google.com>
3 participants
Description
Fixes #4033
Scenarios Tested
Verified that
firebase emulators:export ./emulator-data --only authonly exportsauthdataVerified that
firebase emulators:export ./emulator-dataexports all dataVerified that POST works for exporting
Running below will export all
Running below will export only auth
Sample Commands
firebase emulators:export ./emulator-data --only auth