Skip to content

Add sanitized-string format #5007

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
julienw merged 10 commits into from
Jun 4, 2024
Merged

Add sanitized-string format #5007

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
89e952b
Add sanitized-string format
valenting May 22, 2024
f2ca5cf
fix lint
valenting May 22, 2024
0172c7f
Bump profile version to 30
valenting May 22, 2024
a166cd7
Bump processed profile version to 49
valenting May 22, 2024
4aff239
Fix tests
valenting May 22, 2024
9b09352
Add sanitized-string logic to formatFromMarkerSchema and test
valenting May 22, 2024
1c63a32
Add unit test for sanitized-string fields
valenting May 22, 2024
68b7702
Add sanitized-string to MarkerFormatType
valenting May 22, 2024
03935e4
Fix test lint
valenting May 22, 2024
4fe78ba
missing ;
valenting May 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs-developer/CHANGELOG-formats.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ Note that this is not an exhaustive list. Processed profile format upgraders can

## Processed profile format

### Version 49

A new `sanitized-string` marker schema format type has been added, allowing markers to carry arbitrary strings containing PII that will be sanitized along with URLs and FilePaths.

### Version 48

Removed the 'sampleGroups' object from the Counter structure.
Expand Down Expand Up @@ -70,6 +74,10 @@ Older versions are not documented in this changelog but can be found in [process

## Gecko profile format

### Version 30

A new `sanitized-string` marker schema format type has been added, allowing markers to carry arbitrary strings containing PII that will be sanitized along with URLs and FilePaths.

### Version 29

Removed the 'sample_groups' object from the GeckoCounter structure.
Expand Down
4 changes: 2 additions & 2 deletions src/app-logic/constants.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ import type { MarkerPhase } from 'firefox-profiler/types';
// The current version of the Gecko profile format.
// Please don't forget to update the gecko profile format changelog in
// `docs-developer/CHANGELOG-formats.md`.
export const GECKO_PROFILE_VERSION = 29;
export const GECKO_PROFILE_VERSION = 30;

// The current version of the "processed" profile format.
// Please don't forget to update the processed profile format changelog in
// `docs-developer/CHANGELOG-formats.md`.
export const PROCESSED_PROFILE_VERSION = 48;
export const PROCESSED_PROFILE_VERSION = 49;

// The following are the margin sizes for the left and right of the timeline. Independent
// components need to share these values.
Expand Down
7 changes: 7 additions & 0 deletions src/profile-logic/gecko-profile-versioning.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1473,6 +1473,13 @@ const _upgraders = {
}
convertToVersion29Recursive(profile);
},
[30]: (_) => {
// This version bump added a new marker schema format type, named "sanitized-string",
// which older frontends will not be able to display.
// No upgrade is needed, as older versions of firefox would not generate
// marker data with sanitized-string typed data, and no modification is needed in the
// frontend to display older formats.
},

// If you add a new upgrader here, please document the change in
// `docs-developer/CHANGELOG-formats.md`.
Expand Down
5 changes: 5 additions & 0 deletions src/profile-logic/marker-data.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1464,6 +1464,11 @@ export function sanitizeFromMarkerSchema(
...markerPayload,
[key]: removeFilePath(markerPayload[key]),
}: any);
} else if (format === 'sanitized-string') {
markerPayload = ({
...markerPayload,
[key]: '<sanitized>',
}: any);
}
}
}
Expand Down
1 change: 1 addition & 0 deletions src/profile-logic/marker-schema.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -472,6 +472,7 @@ export function formatFromMarkerSchema(
switch (format) {
case 'url':
case 'file-path':
case 'sanitized-string':
case 'string':
// Make sure a non-empty string is returned here.
return String(value) || '(empty)';
Expand Down
3 changes: 3 additions & 0 deletions src/profile-logic/processed-profile-versioning.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2264,6 +2264,9 @@ const _upgraders = {
}
}
},
[49]: (_) => {
// The 'sanitized-string' marker schema format type has been added.
},
// If you add a new upgrader here, please document the change in
// `docs-developer/CHANGELOG-formats.md`.
};
Expand Down
4 changes: 2 additions & 2 deletions src/test/store/__snapshots__/profile-view.test.js.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -423,15 +423,15 @@ Object {
"oscpu": "",
"physicalCPUs": 0,
"platform": "",
"preprocessedProfileVersion": 48,
"preprocessedProfileVersion": 49,
"processType": 0,
"product": "Firefox",
"sourceURL": "",
"stackwalk": 0,
"startTime": 0,
"symbolicated": true,
"toolkit": "",
"version": 29,
"version": 30,
},
"pages": Array [
Object {
Expand Down
48 changes: 24 additions & 24 deletions src/test/unit/__snapshots__/profile-conversion.test.js.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -469,7 +469,7 @@ Object {
"oscpu": undefined,
"physicalCPUs": undefined,
"platform": undefined,
"preprocessedProfileVersion": 48,
"preprocessedProfileVersion": 49,
"processType": 0,
"product": "ART Trace (Android)",
"sampleUnits": undefined,
Expand All @@ -479,7 +479,7 @@ Object {
"symbolicated": true,
"toolkit": undefined,
"updateChannel": undefined,
"version": 29,
"version": 30,
"visualMetrics": undefined,
},
"pages": Array [],
Expand Down Expand Up @@ -100220,7 +100220,7 @@ Object {
"oscpu": undefined,
"physicalCPUs": undefined,
"platform": undefined,
"preprocessedProfileVersion": 48,
"preprocessedProfileVersion": 49,
"processType": 0,
"product": "ART Trace (Android)",
"sampleUnits": undefined,
Expand All @@ -100230,7 +100230,7 @@ Object {
"symbolicated": true,
"toolkit": undefined,
"updateChannel": undefined,
"version": 29,
"version": 30,
"visualMetrics": undefined,
},
"pages": Array [],
Expand Down Expand Up @@ -382162,15 +382162,15 @@ Object {
"oscpu": "",
"physicalCPUs": 0,
"platform": "",
"preprocessedProfileVersion": 48,
"preprocessedProfileVersion": 49,
"processType": 0,
"product": "Chrome Trace",
"sourceURL": "",
"stackwalk": 0,
"startTime": 0,
"symbolicated": true,
"toolkit": "",
"version": 29,
"version": 30,
},
"pages": Array [],
"threads": Array [
Expand Down Expand Up @@ -430866,15 +430866,15 @@ Object {
"oscpu": "",
"physicalCPUs": 0,
"platform": "",
"preprocessedProfileVersion": 48,
"preprocessedProfileVersion": 49,
"processType": 0,
"product": "Chrome Trace",
"sourceURL": "",
"stackwalk": 0,
"startTime": 0,
"symbolicated": true,
"toolkit": "",
"version": 29,
"version": 30,
},
"pages": Array [],
"threads": Array [
Expand Down Expand Up @@ -479570,7 +479570,7 @@ Object {
"oscpu": "",
"physicalCPUs": 0,
"platform": "",
"preprocessedProfileVersion": 48,
"preprocessedProfileVersion": 49,
"processType": 0,
"product": "Chrome Trace",
"profilingEndTime": 66155012.423,
Expand All @@ -479580,7 +479580,7 @@ Object {
"startTime": 0,
"symbolicated": true,
"toolkit": "",
"version": 29,
"version": 30,
},
"pages": Array [],
"threads": Array [
Expand Down Expand Up @@ -482612,15 +482612,15 @@ Object {
"oscpu": "",
"physicalCPUs": 0,
"platform": "",
"preprocessedProfileVersion": 48,
"preprocessedProfileVersion": 49,
"processType": 0,
"product": "Chrome Trace",
"sourceURL": "",
"stackwalk": 0,
"startTime": 0,
"symbolicated": true,
"toolkit": "",
"version": 29,
"version": 30,
},
"pages": Array [],
"threads": Array [
Expand Down Expand Up @@ -485730,7 +485730,7 @@ Object {
"oscpu": "",
"physicalCPUs": 0,
"platform": "",
"preprocessedProfileVersion": 48,
"preprocessedProfileVersion": 49,
"processType": 0,
"product": "Chrome Trace",
"profilingEndTime": 66155012.423,
Expand All @@ -485740,7 +485740,7 @@ Object {
"startTime": 0,
"symbolicated": true,
"toolkit": "",
"version": 29,
"version": 30,
},
"pages": Array [],
"threads": Array [
Expand Down Expand Up @@ -486930,7 +486930,7 @@ Object {
"oscpu": undefined,
"physicalCPUs": undefined,
"platform": undefined,
"preprocessedProfileVersion": 48,
"preprocessedProfileVersion": 49,
"processType": 0,
"product": "Firefox",
"sampleUnits": undefined,
Expand All @@ -486940,7 +486940,7 @@ Object {
"symbolicated": true,
"toolkit": undefined,
"updateChannel": undefined,
"version": 29,
"version": 30,
"visualMetrics": undefined,
},
"pages": Array [],
Expand Down Expand Up @@ -494455,7 +494455,7 @@ Object {
"oscpu": undefined,
"physicalCPUs": undefined,
"platform": undefined,
"preprocessedProfileVersion": 48,
"preprocessedProfileVersion": 49,
"processType": 0,
"product": "Firefox",
"sampleUnits": undefined,
Expand All @@ -494465,7 +494465,7 @@ Object {
"symbolicated": true,
"toolkit": undefined,
"updateChannel": undefined,
"version": 29,
"version": 30,
"visualMetrics": undefined,
},
"pages": Array [],
Expand Down Expand Up @@ -513868,7 +513868,7 @@ Object {
"oscpu": undefined,
"physicalCPUs": undefined,
"platform": undefined,
"preprocessedProfileVersion": 48,
"preprocessedProfileVersion": 49,
"processType": 0,
"product": "Firefox",
"sampleUnits": undefined,
Expand All @@ -513878,7 +513878,7 @@ Object {
"symbolicated": true,
"toolkit": undefined,
"updateChannel": undefined,
"version": 29,
"version": 30,
"visualMetrics": undefined,
},
"pages": Array [],
Expand Down Expand Up @@ -521386,7 +521386,7 @@ Object {
"oscpu": undefined,
"physicalCPUs": undefined,
"platform": undefined,
"preprocessedProfileVersion": 48,
"preprocessedProfileVersion": 49,
"processType": 0,
"product": "Firefox",
"sampleUnits": undefined,
Expand All @@ -521396,7 +521396,7 @@ Object {
"symbolicated": true,
"toolkit": undefined,
"updateChannel": undefined,
"version": 29,
"version": 30,
"visualMetrics": undefined,
},
"pages": Array [],
Expand Down Expand Up @@ -544986,15 +544986,15 @@ Object {
"oscpu": "",
"physicalCPUs": 0,
"platform": "",
"preprocessedProfileVersion": 48,
"preprocessedProfileVersion": 49,
"processType": 0,
"product": "target/debug/examples/work_log (dhat)",
"sourceURL": "",
"stackwalk": 0,
"startTime": 0,
"symbolicated": true,
"toolkit": "",
"version": 29,
"version": 30,
},
"pages": Array [],
"threads": Array [
Expand Down
14 changes: 7 additions & 7 deletions src/test/unit/__snapshots__/profile-upgrading.test.js.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ Object {
"oscpu": undefined,
"physicalCPUs": undefined,
"platform": undefined,
"preprocessedProfileVersion": 48,
"preprocessedProfileVersion": 49,
"processType": 0,
"product": "Firefox",
"sampleUnits": undefined,
Expand All @@ -50,7 +50,7 @@ Object {
"symbolicated": true,
"toolkit": undefined,
"updateChannel": undefined,
"version": 29,
"version": 30,
"visualMetrics": undefined,
},
"pages": Array [],
Expand Down Expand Up @@ -7975,7 +7975,7 @@ Object {
"stackwalk": 1,
"startTime": 1460221352723.438,
"toolkit": "cocoa",
"version": 29,
"version": 30,
},
"pausedRanges": Array [],
"processes": Array [
Expand Down Expand Up @@ -9431,7 +9431,7 @@ Object {
"stackwalk": 1,
"startTime": 1460221352723.438,
"toolkit": "cocoa",
"version": 29,
"version": 30,
},
"pages": Array [
Object {
Expand Down Expand Up @@ -10971,7 +10971,7 @@ Object {
"misc": "rv:48.0",
"oscpu": "Intel Mac OS X 10.11",
"platform": "Macintosh",
"preprocessedProfileVersion": 48,
"preprocessedProfileVersion": 49,
"processType": 0,
"product": "Firefox",
"stackwalk": 1,
Expand Down Expand Up @@ -12625,7 +12625,7 @@ Object {
"misc": "rv:48.0",
"oscpu": "Intel Mac OS X 10.11",
"platform": "Macintosh",
"preprocessedProfileVersion": 48,
"preprocessedProfileVersion": 49,
"processType": 0,
"product": "Firefox",
"stackwalk": 1,
Expand Down Expand Up @@ -14422,7 +14422,7 @@ Object {
"misc": "rv:48.0",
"oscpu": "Intel Mac OS X 10.11",
"platform": "Macintosh",
"preprocessedProfileVersion": 48,
"preprocessedProfileVersion": 49,
"processType": 0,
"product": "Firefox",
"stackwalk": 1,
Expand Down
2 changes: 2 additions & 0 deletions src/test/unit/marker-schema.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,7 @@ describe('marker schema formatting', function () {
['file-path', '/Users/me/gecko'],
['file-path', null],
['file-path', undefined],
['sanitized-string', 'domain.name'],
['duration', 0],
['duration', 10],
['duration', 12.3456789],
Expand Down Expand Up @@ -298,6 +299,7 @@ describe('marker schema formatting', function () {
"file-path - /Users/me/gecko",
"file-path - (empty)",
"file-path - (empty)",
"sanitized-string - domain.name",
"duration - 0s",
"duration - 10ms",
"duration - 12.346ms",
Expand Down
Loading