Help with Docker on Windows 11 - Only running on a Single CPU instead of both

TheGoldBug · May 4, 2023, 10:36pm

I struggled thru installing Git (so many questions to answer) and Desktop Docker and am able to run WedODM jobs but I can’t get Docker to use both of my CPUs. I have dual 20 core (so 80 threads in total) cpus and edits to “.wslconfig” are read (I know because I change the memory) and take effect but only up to one cpu. Lots of searching seems to indicate that I need to issue a --cpus parameter on the command line that starts the Docker container but so far I’m not sure in which file that resides.

thanks

Orkboy59 · May 4, 2023, 11:48pm

Windows 11 Pro?

TheGoldBug · May 5, 2023, 12:32am

Yes.

Saijin_Naib · May 5, 2023, 3:54am

It looks like WSL2 doesn’t support more than 64 threads, and only one physical CPU.

This issue goes back to 2020 at least, it seems.

github.com/microsoft/WSL

WSL 2 uses half the number of cores on AMD Threadripper 3990X

opened 07:55PM - 16 Jun 20 UTC

AAlMutairi

bug

# Environment ```none Windows build number: Microsoft Windows [Version 10.0.…19041.329] Your Distribution version: Ubuntu: 20.04 WSL 2 ``` # Steps to reproduce I am using AMD threadripper 3990x in my PC. when I use the command lscpu I get the following ```none Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian Address sizes: 48 bits physical, 48 bits virtual CPU(s): 64 On-line CPU(s) list: 0-63 Thread(s) per core: 2 Core(s) per socket: 32 Socket(s): 1 Vendor ID: AuthenticAMD CPU family: 23 Model: 49 Model name: AMD Ryzen Threadripper 3990X 64-Core Processor . . . ``` Also when I use the command nproc, I get 64. However, using both openmpi and mpich to run parallel job, mpi uses only 32 cores (half real cores). For this test I used the following code (copied from: https://mpitutorial.com/tutorials/mpi-hello-world/) ```none #include <mpi.h> #include <stdio.h> int main(int argc, char** argv) { // Initialize the MPI environment MPI_Init(NULL, NULL); // Get the number of processes int world_size; MPI_Comm_size(MPI_COMM_WORLD, &world_size); // Get the rank of the process int world_rank; MPI_Comm_rank(MPI_COMM_WORLD, &world_rank); // Get the name of the processor char processor_name[MPI_MAX_PROCESSOR_NAME]; int name_len; MPI_Get_processor_name(processor_name, &name_len); // Print off a hello world message printf("Hello world from processor %s, rank %d out of %d processors\n", processor_name, world_rank, world_size); // Finalize the MPI environment. MPI_Finalize(); } ```  # Expected behavior ```none . . . Hello world from processor Ubuntu, rank 10 out of 64 processors Hello world from processor Ubuntu, rank 18 out of 64 processors Hello world from processor Ubuntu, rank 23 out of 64 processors . . . ``` # Actual behavior ```none . . . Hello world from processor Ubuntu, rank 10 out of 32 processors Hello world from processor Ubuntu, rank 18 out of 32 processors Hello world from processor Ubuntu, rank 23 out of 32 processors . . . ```

TheGoldBug · May 5, 2023, 5:55am

Thanks for doing the research and checking that out Saijin. Docker and WSL were out of my realm of experience until last night. With the exception of my laptop, all my machines are dual cpu so taking a 50% processing power hit is out of the question for these big jobs. So for my machines it looks like I need to be running Docker on Linux or a Native Windows install to be using all the CPUs and cores.

I did encounter another issue with this Docker on Windows testing shown in the capture below.

Looks like the root drive (c:) disk space is just about all consumed so it appears to be a lack of space issue.

Saijin_Naib · May 5, 2023, 2:27pm

Check the max volume size for your storage vhdx which Docker/WSL2 use, it Defaults to 250GB or so. You can expand it further if you have the room.

And yeah, with this limitation, native Windows or Linux seem best.

system · June 4, 2023, 2:28pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.