Large memory test fails on outerloop

[Drawaes]

/cc @atsushikan

ref dotnet/corefx#24497

From System.Memory.Tests

Suspected test : (thanks @benaadams ) At a guess... would be the ParserTests.2gbOverflow which needs 4 * 2BG > 8 GB to run

Executed on dw7opv3vs0003HH
2017-11-14 20:31:24,865: INFO: scriptrunner(33): _main: BuildTools Helix Script Runner v0.1 starting
2017-11-14 20:31:24,865: INFO: helix_test_execution(24): __init__: Copying execution payload files from C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Work\4b5eacd6-b225-4ea9-a5f7-441d3c2eba3b\Unzip to C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Work\4b5eacd6-b225-4ea9-a5f7-441d3c2eba3b\Exec\execution
2017-11-14 20:31:24,865: INFO: io(27): copy_tree_to: Copying 'C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Work\4b5eacd6-b225-4ea9-a5f7-441d3c2eba3b\Unzip\RunTests.cmd' to 'C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Work\4b5eacd6-b225-4ea9-a5f7-441d3c2eba3b\Exec\execution\RunTests.cmd'
2017-11-14 20:31:24,865: INFO: io(27): copy_tree_to: Copying 'C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Work\4b5eacd6-b225-4ea9-a5f7-441d3c2eba3b\Unzip\System.Memory.Tests.dll' to 'C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Work\4b5eacd6-b225-4ea9-a5f7-441d3c2eba3b\Exec\execution\System.Memory.Tests.dll'
2017-11-14 20:31:24,865: INFO: io(27): copy_tree_to: Copying 'C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Work\4b5eacd6-b225-4ea9-a5f7-441d3c2eba3b\Unzip\System.Memory.Tests.pdb' to 'C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Work\4b5eacd6-b225-4ea9-a5f7-441d3c2eba3b\Exec\execution\System.Memory.Tests.pdb'
2017-11-14 20:31:24,880: INFO: io(27): copy_tree_to: Copying 'C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Work\4b5eacd6-b225-4ea9-a5f7-441d3c2eba3b\Unzip\xunit.console.netcore.exe' to 'C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Work\4b5eacd6-b225-4ea9-a5f7-441d3c2eba3b\Exec\execution\xunit.console.netcore.exe'
2017-11-14 20:31:24,880: INFO: io(27): copy_tree_to: Copying 'C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Work\4b5eacd6-b225-4ea9-a5f7-441d3c2eba3b\Unzip\xunit.console.netcore.runtimeconfig.json' to 'C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Work\4b5eacd6-b225-4ea9-a5f7-441d3c2eba3b\Exec\execution\xunit.console.netcore.runtimeconfig.json'
2017-11-14 20:31:24,880: INFO: proc(23): run_and_log_output: Running: C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Work\4b5eacd6-b225-4ea9-a5f7-441d3c2eba3b\Unzip\RunTests.cmd C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Payload
2017-11-14 20:31:24,880: INFO: proc(26): run_and_log_output: CWD: C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Work\4b5eacd6-b225-4ea9-a5f7-441d3c2eba3b\Unzip
2017-11-14 20:31:24,895: INFO: proc(54): run_and_log_output: Output: Using C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Payload as the test runtime folder.
2017-11-14 20:31:24,895: INFO: proc(54): run_and_log_output: Output: Executing in C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Work\4b5eacd6-b225-4ea9-a5f7-441d3c2eba3b\Unzip\ 
2017-11-14 20:31:24,895: INFO: proc(54): run_and_log_output: Output: Running tests... Start time: 20:31:24.89
2017-11-14 20:31:24,895: INFO: proc(54): run_and_log_output: Output: 
2017-11-14 20:31:24,895: INFO: proc(54): run_and_log_output: Output: C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Work\4b5eacd6-b225-4ea9-a5f7-441d3c2eba3b\Unzip>set XUNIT_PERFORMANCE_MIN_ITERATION=1 
2017-11-14 20:31:24,895: INFO: proc(54): run_and_log_output: Output: 
2017-11-14 20:31:24,895: INFO: proc(54): run_and_log_output: Output: C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Work\4b5eacd6-b225-4ea9-a5f7-441d3c2eba3b\Unzip>set XUNIT_PERFORMANCE_MAX_ITERATION=1 
2017-11-14 20:31:24,895: INFO: proc(54): run_and_log_output: Output: 
2017-11-14 20:31:24,895: INFO: proc(54): run_and_log_output: Output: C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Work\4b5eacd6-b225-4ea9-a5f7-441d3c2eba3b\Unzip>call C:\dotnetbuild\work\0f6bf7e2-4ce4-4832-b62e-da821b97c51b\Payload\dotnet.exe xunit.console.netcore.exe System.Memory.Tests.dll  -xml testResults.xml -notrait category=nonnetcoreapptests -notrait category=nonwindowstests  -notrait category=failing 
2017-11-14 20:31:24,990: INFO: proc(54): run_and_log_output: Output: xUnit.net console test runner (64-bit .NET Core)
2017-11-14 20:31:24,990: INFO: proc(54): run_and_log_output: Output: Copyright (C) 2014 Outercurve Foundation.
2017-11-14 20:31:24,990: INFO: proc(54): run_and_log_output: Output: 
2017-11-14 20:31:25,084: INFO: proc(54): run_and_log_output: Output: Discovering: System.Memory.Tests
2017-11-14 20:31:25,411: INFO: proc(54): run_and_log_output: Output: Discovered:  System.Memory.Tests
2017-11-14 20:31:25,693: INFO: proc(54): run_and_log_output: Output: Starting:    System.Memory.Tests
2017-11-14 20:31:27,661: INFO: proc(54): run_and_log_output: Output: [ParseInt32 '0',D to 0)]
2017-11-14 20:31:48,457: INFO: proc(54): run_and_log_output: Output: [ParseInt32 '2',D to 2)]
2017-11-14 20:32:06,128: INFO: proc(54): run_and_log_output: Output: [ParseInt32 '21',D to 21)]
2017-11-14 20:32:23,223: INFO: proc(54): run_and_log_output: Output: [ParseInt32 '+2',D to 2)]
2017-11-14 20:32:40,924: INFO: proc(54): run_and_log_output: Output: [ParseInt32 '-2',D to -2)]
2017-11-14 20:32:57,706: INFO: proc(54): run_and_log_output: Output: [ParseInt32 '2147483647',D to 2147483647)]
2017-11-14 20:33:14,471: INFO: proc(54): run_and_log_output: Output: [ParseInt32 '-2147483648',D to -2147483648)]
2017-11-14 20:33:31,707: INFO: proc(54): run_and_log_output: Output: [ParseInt32 '2147483648',D to (should-not-parse))]
2017-11-14 20:33:48,144: INFO: proc(54): run_and_log_output: Output: [ParseInt32 '-2147483649',D to (should-not-parse))]
2017-11-14 20:34:00,503: INFO: proc(54): run_and_log_output: Output: [ParseInt32 '12345abcdefg1',D to 12345)]
2017-11-14 20:34:11,926: INFO: proc(54): run_and_log_output: Output: [ParseInt32 '1234145abcdefg1',D to 1234145)]
2017-11-14 20:34:23,176: INFO: proc(54): run_and_log_output: Output: [ParseInt32 'abcdefghijklmnop1',D to 0)]
2017-11-14 20:34:34,815: INFO: proc(54): run_and_log_output: Output: [ParseInt32 '1147483648',D to 1147483648)]
2017-11-14 20:34:46,362: INFO: proc(54): run_and_log_output: Output: [ParseInt32 '-1147483649',D to -1147483649)]

@ahsonkhan - is there still a reason for this to be open?

[stephentoub]

is there still a reason for this to be open?

Yes, tests are still disabled against it.

Can't repro this locally.

Was it hung in one of those two tests or was the hang appearing somewhere else as a result of these two tests existing?

These two tests are the only ones performing two big allocations in a single test, but I can't come up with a theory of why that would be problematic. They're also among the hoggiest tests (8Gb) - could be they're just causing random OM exceptions in some other test that isn't a "big allocation" test?

[benaadams]

Can it use a MemoryMappedFile instead of allocating 8Gb of RAM? Used to give my computer a fit when it was in corefxlab and I ran it:

Can the tests potentially run in parallel so need 16Gb?

[benaadams]

Everything would die on the test with nothing running after; last output when details output was on was:

2017-11-15 15:46:24,285: INFO: proc(54): run_and_log_output: 
Output:    System.SpanTests.SpanTests.CopyToLargeSizeTest(bufferSize: 4294967296) [STARTING]

My local experience was Windows was very unhappy working swap for a single contiguous block of memory larger than available memory. It did eventually complete after total system unresponsiveness

[Drawaes]

Yeah but it also doesn't run in isolation, so it truly kills the test run.

Are we reasonably confident a MemoryMappedFile for 8gb won't cause the same agony as a 8gb memory allocation?

The AllocationHelper serializes big memory tests that uses it so they shouldn't be running in parallel.

[benaadams]

Don't know; however it only pretends to be contiguous memory and files can often be far larger than RAM - so hopefully the disk cache will be more used to it than the swap with actual live allocated memory

@benaadams - if you still have that computer, can you trying building this branch and running SpanTests.CopyToLargeSizeTest to see if switching to memory-mapped files still brings the puter to its knees?

https://github.com/AtsushiKan/corefx/tree/test