Added Memory Information - Free and Total memory for different platforms #847

ChiragRupani · 2018-08-08T17:25:14Z

Added Memory Information - Free and Total memory for different platforms, for #846 Add RAM to summary


          Added Memory Information - Free and Total memory for different platforms


          Added Memory Information - Free and Total memory for different platforms

AndreyAkinshin · 2018-08-09T07:07:16Z

src/BenchmarkDotNet/Portability/Memory/SysctlMemoryInfoProvider.cs

+        {
+            if (RuntimeInformation.IsMacOSX())
+            {
+                string sysctlContent = ProcessHelper.RunAndReadOutput("sysctl", "-a");


We already call sysctl in SysctlCpuInfoProvider. Such calls are pretty expensive. Please, reuse existed data from SysctlCpuInfoProvider instead of an additional call.

Right, added code to reuse call output for CPU and memory

AndreyAkinshin · 2018-08-09T07:09:28Z

src/BenchmarkDotNet/Portability/Memory/wmicMemoryInfoProvider.cs

+        {
+            if (RuntimeInformation.IsWindows())
+            {
+                string content = ProcessHelper.RunAndReadOutput("wmic", "OS get TotalVisibleMemorySize, FreePhysicalMemory /Format:List");


Please, reuse the wmic call from WmicCpuInfoProvider

wmic cpu is used for CPU and wmic OS is used for memory, the call output can not be reused

AndreyAkinshin · 2018-08-09T07:13:27Z

src/BenchmarkDotNet/Portability/RuntimeInformation.cs

@@ -155,6 +156,22 @@ internal static CpuInfo GetCpuInfo()
            return null;
        }

+        public static bool IsVistaAndAbove() => Version.Parse(RuntimeEnvironment.OperatingSystemVersion).Major >= 6;


BenchmarkDotNet v0.11.0+ requires .NET Framework 4.6+. Windows XP doesn't support .NET Framework 4.6+ and .NET Core. So, we can assume that the Windows version is always vista or above.

Removed the code

AndreyAkinshin · 2018-08-09T07:14:43Z

src/BenchmarkDotNet/Portability/RuntimeInformation.cs

+            if (IsWindows() && IsVistaAndAbove())
+                return MoSMemoryInfoProvider.MosMemoryInfo.Value;
+            if (IsWindows())
+                return WmicMemoryInfoProvider.WmicMemoryInfo.Value;


Do we really need two different ways to get this information on Windows?

I guess, it's better to use only Wmic because we already have a wmic call.

AndreyAkinshin · 2018-08-09T07:17:17Z

tests/BenchmarkDotNet.Tests/BenchmarkDotNet.Tests.csproj

+  <ItemGroup>
+    <None Remove="Portability\Memory\TestFiles\SysctlMemory.txt" />
+    <None Remove="Portability\Memory\TestFiles\VmStatMemory.txt" />
+  </ItemGroup>


Why do we need these lines?

Removed - not required.

AndreyAkinshin · 2018-08-09T07:17:51Z

tests/BenchmarkDotNet.Tests/BenchmarkDotNet.Tests.csproj

+  <ItemGroup>
+    <EmbeddedResource Include="Portability\Memory\TestFiles\SysctlMemory.txt" />
+    <EmbeddedResource Include="Portability\Memory\TestFiles\VmStatMemory.txt" />
+  </ItemGroup>


Please, embed the whole folder (<EmbeddedResource Include="Portability\Memory\TestFiles\*.txt" />)

AndreyAkinshin

LGTM in general. Some minor changes are requested.


          Added common SysctlInfoProvider for CPU and Memory Info on MacOSX


          Removed Management object search in favour of wmic

ChiragRupani · 2018-08-09T16:08:33Z

Resolved code review comments, for failing approval tests - total memory need to be of CI machine and free memory is dynamic. How it can be handled?

AndreyAkinshin · 2018-08-09T16:35:32Z

Resolved code review comments, for failing approval tests - total memory need to be of CI machine and free memory is dynamic. How it can be handled?

You should replace it in MockHostEnvironmentInfo by fake hardcoded values.

AndreyAkinshin · 2018-08-09T16:37:41Z

src/BenchmarkDotNet/Portability/SysctlInfoProvider.cs

+    internal static class SysctlInfoProvider
+    {
+        internal static readonly Lazy<string> SysctlInfo = new Lazy<string>(Load);
+        private static string _sysctlInfo = null;


We don't use _ as a field prefix.


          Included Memory Info in Exporter Approval tests


          Fixed memory info for Mac OS

AndreyAkinshin · 2018-08-10T11:23:21Z

I have another concern about Linux. I don't think that it makes sense to print "Free" memory from /proc/meminfo. Currently, I have the following line in the BenchmarkDotNet output:

Total Memory=31.38 GB, Free Memory=0.37 GB

The whole stats (some numbers have a small drift because I had different activities between measurements):

$ cat /proc/meminfo 
MemTotal:       32904420 kB
MemFree:          415612 kB
MemAvailable:    6204328 kB
Buffers:         1270516 kB
Cached:          3843920 kB
SwapCached:         1136 kB
Active:         27942284 kB
Inactive:        3192896 kB
Active(anon):   23296324 kB
Inactive(anon):  2192128 kB
Active(file):    4645960 kB
Inactive(file):  1000768 kB
Unevictable:         232 kB
Mlocked:             232 kB
SwapTotal:       7999484 kB
SwapFree:        7974908 kB
Dirty:              1232 kB
Writeback:             0 kB
AnonPages:      26020568 kB
Mapped:          2779036 kB
Shmem:            969228 kB
Slab:             841104 kB
SReclaimable:     611448 kB
SUnreclaim:       229656 kB
KernelStack:       28480 kB
PageTables:       153144 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    24451692 kB
Committed_AS:   40689380 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:     3029980 kB
DirectMap2M:    29440000 kB
DirectMap1G:     1048576 kB

$ vmstat -s
     32904420 K total memory
     26693376 K used memory
     27852808 K active memory
      3167168 K inactive memory
       545832 K free memory
      1270684 K buffer memory
      4394528 K swap cache
      7999484 K total swap
        24576 K used swap
      7974908 K free swap
     12136084 non-nice user cpu ticks
       287263 nice user cpu ticks
      2815775 system cpu ticks
    192840749 idle cpu ticks
       308475 IO-wait cpu ticks
            0 IRQ cpu ticks
       143378 softirq cpu ticks
            0 stolen cpu ticks
     35812936 pages paged in
    501444032 pages paged out
          475 pages swapped in
         5888 pages swapped out
   1997545531 interrupts
   2354026314 CPU context switches
   1533637428 boot time
      2355729 forks

htop:

top:

I suggest using the htop format and print Mem/used/buffers/cache. What do you think?

AndreyAkinshin · 2018-08-10T11:30:46Z

The same experiment on macOS.
BenchmarkDotNet:

Total Memory=16 GB, Free Memory=0.98 GB

$ vm_stat
Mach Virtual Memory Statistics: (page size of 4096 bytes)
Pages free:                               90662.
Pages active:                           1685717.
Pages inactive:                         1553942.
Pages speculative:                        38418.
Pages throttled:                              0.
Pages wired down:                        682750.
Pages purgeable:                          20762.
"Translation faults":                1314731806.
Pages copy-on-write:                   25196855.
Pages zero filled:                    733593693.
Pages reactivated:                    124220354.
Pages purged:                          13424421.
File-backed pages:                       859068.
Anonymous pages:                        2419009.
Pages stored in compressor:             4872948.
Pages occupied by compressor:            142518.
Decompressions:                       128182734.
Compressions:                         173530019.
Pageins:                               72689687.
Pageouts:                                712547.
Swapins:                               80355093.
Swapouts:                              84599798.

htop:

top:

ChiragRupani · 2018-08-11T07:46:56Z

Linux:
Makes sense and this explains why free memory is always showed as lower value. However, as Used Memory already includes buffer+cache, Can we show: Total/Free/Buffers/Cache?

MacOS:
What about MacOS, which fields need to be considered?

AndreyAkinshin · 2018-08-16T10:35:11Z

@ChiragRupani, I guess we can print "Total/Used" on all operating systems because "Used memory" provides useful information not only on Windows but also on Linux and macOS.

ChiragRupani · 2018-09-03T17:18:46Z

Its better to keep available memory as it is since available memory can be calculated as below:
Linux: MemAvailable
MacOS: Free + Inactive


          Improved Memory Info shown for Linux and MacOS

Linux: MemAvailable MacOS: Free + Inactive


          Changed text Free to Available

ChiragRupani added 2 commits Aug 8, 2018

Added Memory Information - Free and Total memory for different platforms

Loading status checks…

7d7dd02

Added Memory Information - Free and Total memory for different platforms

Loading status checks…

0d41146

AndreyAkinshin reviewed Aug 9, 2018

View changes

AndreyAkinshin requested changes Aug 9, 2018

View changes

ChiragRupani added 2 commits Aug 9, 2018

Added common SysctlInfoProvider for CPU and Memory Info on MacOSX

fe18d29

Removed Management object search in favour of wmic

Loading status checks…

a2c0b08

AndreyAkinshin reviewed Aug 9, 2018

View changes

ChiragRupani added 2 commits Aug 9, 2018

Included Memory Info in Exporter Approval tests

Loading status checks…

42dd62f

Fixed memory info for Mac OS

Loading status checks…

16573bd

ChiragRupani added 2 commits Sep 3, 2018

Improved Memory Info shown for Linux and MacOS

Loading status checks…

8312bb7

Linux: MemAvailable MacOS: Free + Inactive

Changed text Free to Available

Loading status checks…

178e681

Sep	OCT	Nov
	11
2019	2020	2021

dotnet / BenchmarkDotNet

Added Memory Information - Free and Total memory for different platforms #847

Added Memory Information - Free and Total memory for different platforms #847

ChiragRupani commented Aug 8, 2018

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

AndreyAkinshin left a comment

ChiragRupani commented Aug 9, 2018

AndreyAkinshin commented Aug 9, 2018

This comment has been minimized.

This comment has been minimized.

AndreyAkinshin commented Aug 10, 2018

AndreyAkinshin commented Aug 10, 2018

ChiragRupani commented Aug 11, 2018

AndreyAkinshin commented Aug 16, 2018

ChiragRupani commented Sep 3, 2018

dotnet / BenchmarkDotNet

Join GitHub today

Added Memory Information - Free and Total memory for different platforms #847

Added Memory Information - Free and Total memory for different platforms #847

Conversation

ChiragRupani commented Aug 8, 2018

This comment has been minimized.

AndreyAkinshin Aug 9, 2018 Member

This comment has been minimized.

ChiragRupani Aug 9, 2018 Author

This comment has been minimized.

AndreyAkinshin Aug 9, 2018 Member

This comment has been minimized.

ChiragRupani Aug 9, 2018 Author

This comment has been minimized.

AndreyAkinshin Aug 9, 2018 Member

This comment has been minimized.

AndreyAkinshin Aug 9, 2018 Member

This comment has been minimized.

ChiragRupani Aug 9, 2018 Author

This comment has been minimized.

AndreyAkinshin Aug 9, 2018 Member

This comment has been minimized.

AndreyAkinshin Aug 9, 2018 Member

This comment has been minimized.

ChiragRupani Aug 9, 2018 Author

This comment has been minimized.

AndreyAkinshin Aug 9, 2018 Member

This comment has been minimized.

ChiragRupani Aug 9, 2018 Author

This comment has been minimized.

AndreyAkinshin Aug 9, 2018 Member

This comment has been minimized.

ChiragRupani Aug 9, 2018 Author

AndreyAkinshin left a comment

ChiragRupani commented Aug 9, 2018

AndreyAkinshin commented Aug 9, 2018

This comment has been minimized.

AndreyAkinshin Aug 9, 2018 Member

This comment has been minimized.

ChiragRupani Aug 9, 2018 Author

AndreyAkinshin commented Aug 10, 2018

AndreyAkinshin commented Aug 10, 2018

ChiragRupani commented Aug 11, 2018

AndreyAkinshin commented Aug 16, 2018

ChiragRupani commented Sep 3, 2018

Essential cookies

Always active

Analytics cookies

AndreyAkinshin Aug 9, 2018
Member

ChiragRupani Aug 9, 2018
Author

AndreyAkinshin Aug 9, 2018
Member

ChiragRupani Aug 9, 2018
Author

AndreyAkinshin Aug 9, 2018
Member

AndreyAkinshin Aug 9, 2018
Member

ChiragRupani Aug 9, 2018
Author

AndreyAkinshin Aug 9, 2018
Member

AndreyAkinshin Aug 9, 2018
Member

ChiragRupani Aug 9, 2018
Author

AndreyAkinshin Aug 9, 2018
Member

ChiragRupani Aug 9, 2018
Author

AndreyAkinshin Aug 9, 2018
Member

ChiragRupani Aug 9, 2018
Author

AndreyAkinshin Aug 9, 2018
Member

ChiragRupani Aug 9, 2018
Author