Amazon EC2の「Cluster Compute Instances」を使ってみた(High Performance Computing向け)


By Waleed Alzuhair

Amazon EC2(HPC)Cluster Compute Instances




Cluster ComputeEC2
Linux
  - CNET Japan  


HadoopClusterAmazon Elastic MapReduce



サーバスペック






23 GB of memory

33.5 EC2 Compute Units (2 x Intel Xeon X5570, quad-core Nehalem architecture)

1690 GB of instance storage

64-bit platform

I/O Performance: Very High (10 Gigabit Ethernet)

API name: cc1.4xlarge

 High Performance Computing (HPC) in the Cloud | Accelerated Computing  

1$1.6
(Reserved Instances)


起動してみよう


Amazon EC2api-tools()AWS Management Console

AWS Management Console使


5 Windows Server() (Windows Server on Amazon EC2)



Cluster Compute InstancesCentOS(Linux)64EBS(us-east)



HPC(HVM)AMICluster Compute



Placement Group

via. New Amazon EC2 Instance Type  The Cluster Compute Instance | AWS News Blog



(8)


Authenticating with public key "imported-openssh-key"


         __|  __|_  ) CentOS
         _|  (     /    v5.4
        ___|\___|___| HVMx64

  Welcome to an EC2 Public Image
                       :-)


[root@ip-10-17-39-245 ~]#

SSH


色々、見てみよう


"/proc/cpuinfo"16(core)vCPU
"2 x Intel Xeon X5570, quad-core Nehalem architecture"HTon16



# free
             total       used       free     shared    buffers     cached
Mem:      23104316     436576   22667740          0      26004     298176
-/+ buffers/cache:     112396   22991920
Swap:     10125304          0   10125304



# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                      9.9G  3.0G  6.5G  32% /
/dev/hda1              99M   19M   75M  20% /boot
tmpfs                  12G     0   12G   0% /dev/shm

"/dev/sdb""/dev/sdc"845GB2(mount使)


ネットワークを軽くベンチマーク


I/O Performance: Very High (10 Gigabit Ethernet)Cluster Compute Instance2Placement Groupiperf


# iperf -c 10.17.33.247
------------------------------------------------------------
Client connecting to 10.17.33.247, TCP port 5001
TCP window size: 27.7 KByte (default)
------------------------------------------------------------
[3] local 10.17.39.245 port 47745 connected with 10.17.33.247 port 5001
[ID] Interval       Transfer     Bandwidth
[3]  0.0-10.0 sec  11.0 GBytes  9.45 Gbits/sec

9.45 Gbits/sec10G


# iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[4] local 10.17.33.247 port 5001 connected with 10.17.39.245 port 47745
[ID] Interval       Transfer     Bandwidth
[4]  0.0-10.0 sec  11.0 GBytes  9.49 Gbits/sec

()


ディスクを軽くベンチマーク

これもあくまで目安ですけど、hdparmでチェック。

/dev/mapper/VolGroup00-LogVol00

まずは、ルートデバイスから確認。

# hdparm -t /dev/mapper/VolGroup00-LogVol00
/dev/mapper/VolGroup00-LogVol00:
 Timing buffered disk reads:  402 MB in  3.04 seconds = 132.26 MB/sec

# hdparm -t /dev/mapper/VolGroup00-LogVol00
/dev/mapper/VolGroup00-LogVol00:
 Timing buffered disk reads:  434 MB in  3.00 seconds = 144.55 MB/sec

# hdparm -t /dev/mapper/VolGroup00-LogVol00
/dev/mapper/VolGroup00-LogVol00:
 Timing buffered disk reads:  460 MB in  3.00 seconds = 153.23 MB/sec
/dev/sdb

次にエフェメラルディスク分。結構はやい。

# hdparm -t /dev/sdb
/dev/sdb:
 Timing buffered disk reads:  786 MB in  3.00 seconds = 261.79 MB/sec

# hdparm -t /dev/sdb
/dev/sdb:
 Timing buffered disk reads:  770 MB in  3.01 seconds = 255.61 MB/sec

# hdparm -t /dev/sdb
/dev/sdb:
 Timing buffered disk reads:  818 MB in  3.00 seconds = 272.35 MB/sec
/dev/sdc

一応、もう一本の方のエフェメラルディスク。たまたまだと思うけど、気持ち少しはやいような?

# hdparm -t /dev/sdc
/dev/sdc:
 Timing buffered disk reads:  878 MB in  3.00 seconds = 292.56 MB/sec

# hdparm -t /dev/sdc
/dev/sdc:
 Timing buffered disk reads:  814 MB in  3.00 seconds = 271.21 MB/sec

# hdparm -t /dev/sdc
/dev/sdc:
 Timing buffered disk reads:  878 MB in  3.00 seconds = 292.46 MB/sec

CPUを軽くベンチマーク





Linpack88041.821418200Top50014541.88
  - CNET Japan  



"C, static allocate version"M
via. http://accc.riken.jp/HPC/HimenoBMT/download1.html#list3

# ./bmt
mimax = 129 mjmax = 129 mkmax = 257
imax = 128 jmax = 128 kmax =256
 Start rehearsal measurement process.
 Measure the performance in 3 times.

 MFLOPS: 1645.656590 time(s): 0.249940 1.733593e-03

 Now, start the actual measurement process.
 The loop will be excuted in 720 times
 This will take about one minute.
 Wait for a while

 Loop executed for 720 times
 Gosa : 8.707814e-04
 MFLOPS measured : 1756.367102  cpu : 56.204479
 Score based on Pentium III 600MHz : 21.419111




付録

最後ということで、ちょっと長くなりそうなものを、以下に貼り付けておきます。

/proc/version

# cat /proc/version
Linux version 2.6.18-164.15.1.el5 (mockbuild@builder10.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)) #1 SMP Wed Mar 17 11:30:06 EDT 2010

/proc/meminfo

# cat /proc/meminfo
MemTotal:     23104316 kB
MemFree:      22667864 kB
Buffers:         26004 kB
Cached:         298176 kB
SwapCached:          0 kB
Active:          85544 kB
Inactive:       288348 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     23104316 kB
LowFree:      22667864 kB
SwapTotal:    10125304 kB
SwapFree:     10125304 kB
Dirty:               0 kB
Writeback:           0 kB
AnonPages:       49760 kB
Mapped:          19180 kB
Slab:            26596 kB
PageTables:       5312 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  21677460 kB
Committed_AS:   139824 kB
VmallocTotal: 34359738367 kB
VmallocUsed:      5252 kB
VmallocChunk: 34359733031 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

/proc/cpuinfo

# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           X5570  @ 2.93GHz
stepping        : 5
cpu MHz         : 2933.445
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 0
cpu cores       : 4
apicid          : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc pni ssse3 cx16 sse4_1 sse4_2 popcnt lahf_lm
bogomips        : 5866.89
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           X5570  @ 2.93GHz
stepping        : 5
cpu MHz         : 2933.445
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 1
cpu cores       : 4
apicid          : 2
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc pni ssse3 cx16 sse4_1 sse4_2 popcnt lahf_lm
bogomips        : 5867.42
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 2
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           X5570  @ 2.93GHz
stepping        : 5
cpu MHz         : 2933.445
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 2
cpu cores       : 4
apicid          : 4
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc pni ssse3 cx16 sse4_1 sse4_2 popcnt lahf_lm
bogomips        : 5866.01
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           X5570  @ 2.93GHz
stepping        : 5
cpu MHz         : 2933.445
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 3
cpu cores       : 4
apicid          : 6
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc pni ssse3 cx16 sse4_1 sse4_2 popcnt lahf_lm
bogomips        : 5867.03
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 4
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           X5570  @ 2.93GHz
stepping        : 5
cpu MHz         : 2933.445
cache size      : 8192 KB
physical id     : 1
siblings        : 8
core id         : 0
cpu cores       : 4
apicid          : 16
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc pni ssse3 cx16 sse4_1 sse4_2 popcnt lahf_lm
bogomips        : 5865.85
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 5
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           X5570  @ 2.93GHz
stepping        : 5
cpu MHz         : 2933.445
cache size      : 8192 KB
physical id     : 1
siblings        : 8
core id         : 1
cpu cores       : 4
apicid          : 18
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc pni ssse3 cx16 sse4_1 sse4_2 popcnt lahf_lm
bogomips        : 5867.53
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 6
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           X5570  @ 2.93GHz
stepping        : 5
cpu MHz         : 2933.445
cache size      : 8192 KB
physical id     : 1
siblings        : 8
core id         : 2
cpu cores       : 4
apicid          : 20
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc pni ssse3 cx16 sse4_1 sse4_2 popcnt lahf_lm
bogomips        : 5867.07
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 7
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           X5570  @ 2.93GHz
stepping        : 5
cpu MHz         : 2933.445
cache size      : 8192 KB
physical id     : 1
siblings        : 8
core id         : 3
cpu cores       : 4
apicid          : 22
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc pni ssse3 cx16 sse4_1 sse4_2 popcnt lahf_lm
bogomips        : 5867.69
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 8
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           X5570  @ 2.93GHz
stepping        : 5
cpu MHz         : 2933.445
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 0
cpu cores       : 4
apicid          : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc pni ssse3 cx16 sse4_1 sse4_2 popcnt lahf_lm
bogomips        : 5866.96
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 9
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           X5570  @ 2.93GHz
stepping        : 5
cpu MHz         : 2933.445
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 1
cpu cores       : 4
apicid          : 3
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc pni ssse3 cx16 sse4_1 sse4_2 popcnt lahf_lm
bogomips        : 5860.77
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 10
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           X5570  @ 2.93GHz
stepping        : 5
cpu MHz         : 2933.445
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 2
cpu cores       : 4
apicid          : 5
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc pni ssse3 cx16 sse4_1 sse4_2 popcnt lahf_lm
bogomips        : 5866.06
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 11
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           X5570  @ 2.93GHz
stepping        : 5
cpu MHz         : 2933.445
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 3
cpu cores       : 4
apicid          : 7
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc pni ssse3 cx16 sse4_1 sse4_2 popcnt lahf_lm
bogomips        : 5866.09
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 12
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           X5570  @ 2.93GHz
stepping        : 5
cpu MHz         : 2933.445
cache size      : 8192 KB
physical id     : 1
siblings        : 8
core id         : 0
cpu cores       : 4
apicid          : 17
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc pni ssse3 cx16 sse4_1 sse4_2 popcnt lahf_lm
bogomips        : 5866.11
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 13
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           X5570  @ 2.93GHz
stepping        : 5
cpu MHz         : 2933.445
cache size      : 8192 KB
physical id     : 1
siblings        : 8
core id         : 1
cpu cores       : 4
apicid          : 19
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc pni ssse3 cx16 sse4_1 sse4_2 popcnt lahf_lm
bogomips        : 5866.46
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 14
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           X5570  @ 2.93GHz
stepping        : 5
cpu MHz         : 2933.445
cache size      : 8192 KB
physical id     : 1
siblings        : 8
core id         : 2
cpu cores       : 4
apicid          : 21
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc pni ssse3 cx16 sse4_1 sse4_2 popcnt lahf_lm
bogomips        : 5866.09
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 15
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           X5570  @ 2.93GHz
stepping        : 5
cpu MHz         : 2933.445
cache size      : 8192 KB
physical id     : 1
siblings        : 8
core id         : 3
cpu cores       : 4
apicid          : 23
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc pni ssse3 cx16 sse4_1 sse4_2 popcnt lahf_lm
bogomips        : 5866.45
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

topコマンドを打つと・・・

壮観です。16個!

top - 05:46:25 up 24 min,  1 user,  load average: 0.66, 0.67, 0.80
Tasks: 231 total,   1 running, 230 sleeping,   0 stopped,   0 zombie
Cpu0  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu4  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu5  :  0.0%us,  0.0%sy,  0.0%ni, 99.0%id,  0.0%wa,  0.0%hi,  1.0%si,  0.0%st
Cpu6  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu7  :  0.3%us,  0.0%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu8  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu9  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu10 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu11 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu12 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu13 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu14 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu15 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  23104316k total,   762708k used, 22341608k free,    17392k buffers
Swap: 10125304k total,        0k used, 10125304k free,   613964k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
    1 root      18   0 10348  684  576 S  0.0  0.0   0:01.74 init
    2 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 migration/0
    3 root      34  19     0    0    0 S  0.0  0.0   0:00.00 ksoftirqd/0
    4 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 watchdog/0
    5 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 migration/1

・・・・・




まとめ


クラウドAMAZON EC2/S3のすべて (ITpro BOOKs)

クラウドAMAZON EC2/S3のすべて (ITpro BOOKs)