Print web link

[bootstraponline]

Fixes #645

Checklist

• Unit tested
• release_notes.md updated

  4 matrix ids created in 0m 22s
  https://console.developers.google.com/storage/browser/test-lab-v9cn46bb990nx-kz69ymd4nm9aq/2020-04-01_07-25-03.132000_AQWv/

Matrices webLink
  matrix-26tm1277rg3gr: https://console.firebase.google.com/project/flank-open-source/testlab/histories/bh.da0c237aaa33732/matrices/8849499259998194241
  matrix-2veeo7xug1f8k: https://console.firebase.google.com/project/flank-open-source/testlab/histories/bh.da0c237aaa33732/matrices/5106500884348802256
  matrix-2z6o0hu8tt5bf: https://console.firebase.google.com/project/flank-open-source/testlab/histories/bh.da0c237aaa33732/matrices/4681223845649672500
  matrix-2jl1g9zptcs9m: https://console.firebase.google.com/project/flank-open-source/testlab/histories/bh.da0c237aaa33732/matrices/6721963817674217321/executions/bs.faacc05c891cc43c

PollMatrices

[codecov-io]

Codecov Report

Merging #683 into master will increase coverage by 0.07%.
The diff coverage is 77.77%.

@@             Coverage Diff              @@
##             master     #683      +/-   ##
============================================
+ Coverage     77.26%   77.33%   +0.07%     
  Complexity      633      633              
============================================
  Files           110      110              
  Lines          2555     2563       +8     
  Branches        364      365       +1     
============================================
+ Hits           1974     1982       +8     
+ Misses          348      347       -1     
- Partials        233      234       +1     

[bootstraponline]

Is there a cleaner way to print the web link once per matrix?

[pawelpasterz]

I am wondering if we could print this link using this logic in SavedMatrix

flank/test_runner/src/main/kotlin/ftl/json/SavedMatrix.kt

Lines 65 to 67 in d91f778

	if (changedLink) {
	this.webLink = newLink
	}

AFAIK link is updated only once for each matrix (please confirm that, I've tested it with couple of examples but might be some corner cases about which I am not aware of).
Also, I think we could print it in more user friendly way -- with indention, message saying what exactly this is etc.
Just thinking, let me know what you think.

[bootstraponline]

I like that idea!

My understanding is the matrix link is only updated once. I'll update the pull request.

[jan-goral]

If we need to print web link ASAP "after starting run", I guess that the cleaner way is to do first refresh of matrices somewhere here. So the web link can be properly generated and printed before "PollMatrices" message will appears. But it's not good idea if number of additional pulls matters.

[jan-goral]

Great job!

[bootstraponline]

A matrix is not guaranteed to have a webLink. A matrix can be invalid.
https://github.com/Flank/flank/blob/master/test_runner/src/main/kotlin/ftl/util/MatrixState.kt

Here are the possible states:

        "enumDescriptions": [
            "Do not use.  For proto versioning only.",
            "The execution or matrix is being validated.",
            "The execution or matrix is waiting for resources to become available.",
            "The execution is currently being processed.\n\nCan only be set on an execution.",
            "The execution or matrix has terminated normally.\n\nOn a matrix this means that the matrix level processing completed normally,\nbut individual executions may be in an ERROR state.",
            "The execution or matrix has stopped because it encountered an\ninfrastructure failure.",
            "The execution was not run because it corresponds to a unsupported\nenvironment.\n\nCan only be set on an execution.",
            "The execution was not run because the provided inputs are incompatible with\nthe requested environment.\n\nExample: requested AndroidVersion is lower than APK's minSdkVersion\n\nCan only be set on an execution.",
            "The execution was not run because the provided inputs are incompatible with\nthe requested architecture.\n\nExample: requested device does not support running the native code in\nthe supplied APK\n\nCan only be set on an execution.",
            "The user cancelled the execution.\n\nCan only be set on an execution.",
            "The execution or matrix was not run because the provided inputs are not\nvalid.\n\nExamples: input file is not of the expected type, is malformed\/corrupt, or\nwas flagged as malware"
          ],
          "type": "string",
          "enum": [
            "TEST_STATE_UNSPECIFIED",
            "VALIDATING",
            "PENDING",
            "RUNNING",
            "FINISHED",
            "ERROR",
            "UNSUPPORTED_ENVIRONMENT",
            "INCOMPATIBLE_ENVIRONMENT",
            "INCOMPATIBLE_ARCHITECTURE",
            "CANCELLED",
            "INVALID"
          ]

• MatrixState.kt
• testing_v1.json#L171

I think only:

"VALIDATING",
"PENDING",
"RUNNING",
"FINISHED",

will possibly have a webLink that we can poll for.

"TEST_STATE_UNSPECIFIED",
"UNSUPPORTED_ENVIRONMENT",
"INCOMPATIBLE_ENVIRONMENT",
"INCOMPATIBLE_ARCHITECTURE",
"CANCELLED",
"INVALID"

I don't think we'll get a web link from these states. In the current implementation, will we poll indefinitely until a web link is provided?

[pawelpasterz]

Thanks for input. I will implement additional logic to prevent this.