docker run mcr.microsoft.com/windows/servercore:ltsc2022 -i --isolation=hyperv resulting in errors

[jt0dd]

Description

According to Docker's documentation, using --isolation=hyperv should make most versions of windows images compatible:

So it seems like compatibility shouldn't be an issue with hyper-v.

Well, when I run:

    docker run mcr.microsoft.com/windows/servercore:ltsc2022 -i --isolation=hyperv

Describe the results you received:

I get the error (see the bottom of the thread, I'm aware the top is just a matter of the image being pulled):

    docker : Unable to find image 'mcr.microsoft.com/windows/servercore:ltsc2022' locally
    At line:1 char:1
    + docker run mcr.microsoft.com/windows/servercore:ltsc2022 -i --isolati ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : NotSpecified: (Unable to find ...sc2022' locally:String) [], RemoteException
        + FullyQualifiedErrorId : NativeCommandError
     
    ltsc2022: Pulling from windows/servercore
    8f616e6e9eec: Pulling fs layer
    037d5740b404: Pulling fs layer
    037d5740b404: Verifying Checksum
    037d5740b404: Download complete
    8f616e6e9eec: Verifying Checksum
    8f616e6e9eec: Download complete
    8f616e6e9eec: Pull complete
    037d5740b404: Pull complete
    Digest: sha256:343f8c95ec604913c74597f6d541e388f83bc0ce373ee8ea5bb26d268879f338
    Status: Downloaded newer image for mcr.microsoft.com/windows/servercore:ltsc2022
    docker: Error response from daemon: container 597ea2d8b5564b2a8771980c3c809b6e059d00d8bc831e97b3cf14a071820aed encountered an error during hcsshim::System::CreateProcess: failure in a Windows system call: The system cannot find 
    the file specified. (0x2)
    [Event Detail:  Provider: 00000000-0000-0000-0000-000000000000]
    [Event Detail:  Provider: 00000000-0000-0000-0000-000000000000]
    [Event Detail: onecore\vm\compute\management\orchestration\vmhostedcontainer\processmanagement.cpp(173)\vmcomputeagent.exe!00007FF61BE0EFBD: (caller: 00007FF61BDB4E97) Exception(2) tid(3c8) 80070002 The system cannot find the 
    file specified.
        CallContext:[\Bridge_ProcessMessage\VmHostedContainer_ExecuteProcess] 
     Provider: 00000000-0000-0000-0000-000000000000].

This is after I've tried many different images, with and without hyper-v, restarted my PC, and reset Docker to factory default settings.

Removing --isolation=hyperv results in precisely the same error. I've also tried --isolation=process and gotten the same error.

Describe the results you expected:

A Windows shell

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version:

    Client:
     Cloud integration: v1.0.22
     Version:           20.10.13
     API version:       1.41
     Go version:        go1.16.15
     Git commit:        a224086
     Built:             Thu Mar 10 14:13:04 2022
     OS/Arch:           windows/amd64
     Context:           default
     Experimental:      true
    
    Server: Docker Desktop 4.6.1 (76265)
     Engine:
      Version:          20.10.13
      API version:      1.41 (minimum version 1.24)
      Go version:       go1.16.15
      Git commit:       906f57f
      Built:            Thu Mar 10 14:08:21 2022
      OS/Arch:          windows/amd64
      Experimental:     false

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.8.1)
  compose: Docker Compose (Docker Inc., v2.3.3)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 3
  Running: 0
  Paused: 0
  Stopped: 3
 Images: 4
 Server Version: 20.10.13
 Storage Driver: windowsfilter
  Windows: 
 Logging Driver: json-file
 Plugins:
  Volume: local
  Network: ics internal l2bridge l2tunnel nat null overlay private transparent
  Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog
 Swarm: inactive
 Default Isolation: hyperv
 Kernel Version: 10.0 22000 (22000.1.amd64fre.co_release.210604-1628)
 Operating System: Windows 10 Pro Version 2009 (OS Build 22000.556)
 OSType: windows
 Architecture: x86_64
 CPUs: 16
 Total Memory: 31.92GiB
 Name: DESKTOP-4KS9BL7
 ID: CVZW:NI3U:JZC6:N763:N5AR:34KN:3QAO:KGIQ:SYDS:WODV:YDGK:U2X7
 Docker Root Dir: C:\ProgramData\Docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

Additional environment details (AWS, VirtualBox, physical, etc.):

I'm running a valid, licensed copy of Windows 11 Pro on a physical PC.

[jt0dd]

I just noticed that if I run the image directly from the Docker for Windows CLI, it works. I get a shell, no problems. So why could the CLI be broken?

[jt0dd]

I figured out the issue. All of this testing has been from a PowerShell ISE terminal. And that's creating the issue somehow. From any other terminal, it actually works.

Edit: No it doesn't. It just doesn't give that error. But it produces a broken container. GUI still works. CLI doesn't, no matter what terminal I run it from. I just get:

PS C:\WINDOWS\system32> docker run mcr.microsoft.com/windows/servercore:ltsc2022
Microsoft Windows [Version 10.0.20348.587]
(c) Microsoft Corporation. All rights reserved.

C:\>
PS C:\WINDOWS\system32> <--- switches back to host system shell

If I try to run the resulting container from the Docker GUI:

It does the same thing, instantly dies.

---

Edit 2: Figured out what's going on here.

One decidedly ignored bug (PowerShell ISE fails with error handling that doesn't shed any light on the actual problem, that's one issue) and a documentation inconsistency. That's what all this comes down to. If I run the command documented here (with -it) it runs fine. I was running the command docker run mcr.microsoft.com/windows/nanoserver:2022 documented here. It says:

The default entry-point is for this image is Cmd.exe. To run the image:

docker run mcr.microsoft.com/windows/nanoserver:2022

I would argue this documentation is just plain wrong, or at the very least misleading.

So first I was running it from the wrong place, and then I was running the wrong command, which the Docker page for that image says to use.

[jt0dd]

retracted

[jt0dd]

Ok, the plot thickens.

I don't mean to complicate things, but each time I think it's working, I find a different thing causing a problem.

This works:
docker run -it mcr.microsoft.com/windows:20H2 --isolation=hyperv

This doesn't:
docker run -it mcr.microsoft.com/windows:20H2 --isolation=hyperv --name=workstation_1

When I add the --name workstation_1 argument, which according to docker run --help is a valid argument:

I get this error:

docker: Error response from daemon: container 1118d316b0700307943bec10392b39266b79db08350311618fb5bfc247e2fa8d encountered an error during hcsshim::System::CreateProcess: failure in a Windows system call: The system cannot find the file specified. (0x2)
[Event Detail:  Provider: 00000000-0000-0000-0000-000000000000]
[Event Detail:  Provider: 00000000-0000-0000-0000-000000000000]
[Event Detail: onecore\vm\compute\management\orchestration\vmhostedcontainer\processmanagement.cpp(173)\vmcomputeagent.exe!00007FF6EBB1A40B: (caller: 00007FF6EBAD5C8B) Exception(1) tid(398) 80070002 The system cannot find the file specified.
    CallContext:[\Bridge_ProcessMessage\VmHostedContainer_ExecuteProcess]
 Provider: 00000000-0000-0000-0000-000000000000].

[thaJeztah]

Looks like the same cause as your other ticket #3549

As to documentation; documentation of images on docker hub is maintained by the respective image publishers (in this case Microsoft); if there's issues with that documentation, it's best to report it via the link mentioned under the "support" section on that page.

[jt0dd]

Looks like the same cause as your other ticket #3549

As to documentation; documentation of images on docker hub is maintained by the respective image publishers (in this case Microsoft); if there's issues with that documentation, it's best to report it via the link mentioned under the "support" section on that page.

I see. I thought since Microsoft had integrated Docker a bit they were working together on that resource. But you're correct. All of this was a rather confusing error message due to an out-of-order argument. In most CLIs can't the argument orders (especially named arguments), be used more freely? Would it be so hard for Docker CLI to just do better parsing of the the arguments passed to it? It isn't exactly difficult to parse --flag=value from a string after all.

[thaJeztah]

In most CLIs can't the argument orders (especially named arguments), be used more freely?

The POSIX guidelines generally recommends flags (options) to appear before positional arguments (operands). While tools often are somewhat more flexible, for example, while on docker inspect the flags should be used before the image name, the cli also accepts them to be reversed;

docker inspect --format="{{.ID}}" busybox:latest

docker inspect busybox:latest --format="{{.ID}}" 

This can only work for some commands though, and was not intentional (but has worked in the past, so we kept the "feature" to not unnecessarily break users), but we don't document (nor officially support) this.

Would it be so hard for Docker CLI to just do better parsing of the arguments passed to it?

... for example, on docker run, docker create, and docker exec the order cannot be reversed, because the position where they appear has a different meaning.

For example, on a Linux container with /bin/busybox as entrypoint, the --help flag after the image name is handled by busybox in the container;

$ docker run --rm --entrypoint=/bin/busybox busybox:latest --help
BusyBox v1.34.1 (2022-03-10 23:58:34 UTC) multi-call binary.
...
...

Putting the --help flag before the image name, means it's an option to handle by the docker run command:

$ docker run --rm --entrypoint=/bin/busybox --help busybox:latest
Usage:  docker run [OPTIONS] IMAGE [COMMAND] [ARG...]

Run a command in a new container
...

It isn't exactly difficult to parse --flag=value from a string after all.

Don't under-estimate the complexity involved in parsing command-line options 😅; command-line parsing can be very complicated; there's many corner-cases (see https://twitter.com/thaJeztah/status/1260921988102656002?s=20 and urfave/cli#1092 for some fun), and depending on the platform or executable in use, sometimes literally impossible (see opencontainers/runtime-spec#998 for some discussion on that (and, yes, that pull request came from Microsoft, and no, they were not able to fix it in Windows itself)).

[jt0dd]

Thanks for the explanations, clearly this is a subject I was (am) incredibly naive on.