Skip to content

Commit 2433734

Browse files
authored
accept memory and gpu memory in GiB instead of MiB (#36)
* accept memory and gpu memory in GiB instead of MiB * implement byte quantity pkg * update cli and selector pkg to byte quantity for memory filters * fixes based on pr comments
1 parent 42ea818 commit 2433734

File tree

19 files changed

+830
-157
lines changed

19 files changed

+830
-157
lines changed

README.md

Lines changed: 27 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ $ export AWS_REGION="us-east-1"
6868

6969
**Find Instance Types with 4 GiB of memory, 2 vcpus, and runs on the x86_64 CPU architecture**
7070
```
71-
$ ec2-instance-selector --memory 4096 --vcpus 2 --cpu-architecture x86_64 -r us-east-1
71+
$ ec2-instance-selector --memory 4 --vcpus 2 --cpu-architecture x86_64 -r us-east-1
7272
c5.large
7373
c5d.large
7474
t2.medium
@@ -93,26 +93,27 @@ r5n.24xlarge
9393

9494
**Short Table Output**
9595
```
96-
$ ec2-instance-selector --memory 4096 --vcpus 2 --cpu-architecture x86_64 -r us-east-1 -o table
97-
Instance Type VCPUs Mem (MiB)
96+
$ ec2-instance-selector --memory 4 --vcpus 2 --cpu-architecture x86_64 -r us-east-1 -o table
97+
Instance Type VCPUs Mem (GiB)
9898
------------- ----- ---------
99-
c5.large 2 4096
100-
c5d.large 2 4096
101-
t2.medium 2 4096
102-
t3.medium 2 4096
103-
t3a.medium 2 4096
99+
c5.large 2 4.000
100+
c5d.large 2 4.000
101+
t2.medium 2 4.000
102+
t3.medium 2 4.000
103+
t3a.medium 2 4.000
104104
```
105105

106106
**Wide Table Output**
107107
```
108-
$ ec2-instance-selector --memory 4096 --vcpus 2 --cpu-architecture x86_64 -r us-east-1 -o table-wide
109-
Instance Type VCPUs Mem (MiB) Hypervisor Current Gen Hibernation Support CPU Arch Network Performance ENIs GPUs
110-
------------- ----- --------- ---------- ----------- ------------------- -------- ------------------- ---- ----
111-
c5.large 2 4096 nitro true true x86_64 Up to 10 Gigabit 3 0
112-
c5d.large 2 4096 nitro true false x86_64 Up to 10 Gigabit 3 0
113-
t2.medium 2 4096 xen true true i386, x86_64 Low to Moderate 3 0
114-
t3.medium 2 4096 nitro true false x86_64 Up to 5 Gigabit 3 0
115-
t3a.medium 2 4096 nitro true false x86_64 Up to 5 Gigabit 3 0
108+
$ ec2-instance-selector --memory 4 --vcpus 2 --cpu-architecture x86_64 -r us-east-1 -o table-wide
109+
Instance Type VCPUs Mem (GiB) Hypervisor Current Gen Hibernation Support CPU Arch Network Performance ENIs GPUs
110+
------------- ----- --------- ---------- ----------- ------------------- -------- ------------------- ---- ----
111+
c5.large 2 4.000 nitro true true x86_64 Up to 10 Gigabit 3 0
112+
c5a.large 2 4.000 nitro true false x86_64 Up to 10 Gigabit 3 0
113+
c5d.large 2 4.000 nitro true false x86_64 Up to 10 Gigabit 3 0
114+
t2.medium 2 4.000 xen true true i386, x86_64 Low to Moderate 3 0
115+
t3.medium 2 4.000 nitro true false x86_64 Up to 5 Gigabit 3 0
116+
t3a.medium 2 4.000 nitro true false x86_64 Up to 5 Gigabit 3 0
116117
```
117118

118119
**All CLI Options**
@@ -144,17 +145,17 @@ Filter Flags:
144145
--deny-list string List of instance types which should be excluded w/ regex syntax (Example: m[1-2]\.*)
145146
-e, --ena-support Instance types where ENA is supported or required
146147
-f, --fpga-support FPGA instance types
147-
--gpu-memory-total int Number of GPUs' total memory in MiB (Example: 4096) (sets --gpu-memory-total-min and -max to the same value)
148-
--gpu-memory-total-max int Maximum Number of GPUs' total memory in MiB (Example: 4096) If --gpu-memory-total-min is not specified, the lower bound will be 0
149-
--gpu-memory-total-min int Minimum Number of GPUs' total memory in MiB (Example: 4096) If --gpu-memory-total-max is not specified, the upper bound will be infinity
148+
--gpu-memory-total float Number of GPUs' total memory in GiB (Example: 4) (sets --gpu-memory-total-min and -max to the same value)
149+
--gpu-memory-total-max float Maximum Number of GPUs' total memory in GiB (Example: 4) If --gpu-memory-total-min is not specified, the lower bound will be 0
150+
--gpu-memory-total-min float Minimum Number of GPUs' total memory in GiB (Example: 4) If --gpu-memory-total-max is not specified, the upper bound will be infinity
150151
-g, --gpus int Total Number of GPUs (Example: 4) (sets --gpus-min and -max to the same value)
151152
--gpus-max int Maximum Total Number of GPUs (Example: 4) If --gpus-min is not specified, the lower bound will be 0
152153
--gpus-min int Minimum Total Number of GPUs (Example: 4) If --gpus-max is not specified, the upper bound will be infinity
153154
--hibernation-support Hibernation supported
154155
--hypervisor string Hypervisor: [xen or nitro]
155-
-m, --memory int Amount of Memory available in MiB (Example: 4096) (sets --memory-min and -max to the same value)
156-
--memory-max int Maximum Amount of Memory available in MiB (Example: 4096) If --memory-min is not specified, the lower bound will be 0
157-
--memory-min int Minimum Amount of Memory available in MiB (Example: 4096) If --memory-max is not specified, the upper bound will be infinity
156+
-m, --memory float Amount of Memory available in GiB (Example: 4) (sets --memory-min and -max to the same value)
157+
--memory-max float Maximum Amount of Memory available in GiB (Example: 4) If --memory-min is not specified, the lower bound will be 0
158+
--memory-min float Minimum Amount of Memory available in GiB (Example: 4) If --memory-max is not specified, the upper bound will be infinity
158159
--network-interfaces int Number of network interfaces (ENIs) that can be attached to the instance (sets --network-interfaces-min and -max to the same value)
159160
--network-interfaces-max int Maximum Number of network interfaces (ENIs) that can be attached to the instance If --network-interfaces-min is not specified, the lower bound will be 0
160161
--network-interfaces-min int Minimum Number of network interfaces (ENIs) that can be attached to the instance If --network-interfaces-max is not specified, the upper bound will be infinity
@@ -221,10 +222,10 @@ func main() {
221222
LowerBound: 2,
222223
UpperBound: 4,
223224
}
224-
// Instantiate an int range filter to specify min and max memory in MiB
225-
memoryRange := selector.IntRangeFilter{
226-
LowerBound: 1024,
227-
UpperBound: 4096,
225+
// Instantiate a float64 range filter to specify min and max memory in GiB
226+
memoryRange := selector.Float64RangeFilter{
227+
LowerBound: 1.0,
228+
UpperBound: 4.0,
228229
}
229230
// Create a string for the CPU Architecture so that it can be passed as a pointer
230231
// when creating the Filter struct

cmd/examples/example1.go

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ package main
33
import (
44
"fmt"
55

6+
"github.com/aws/amazon-ec2-instance-selector/pkg/bytequantity"
67
"github.com/aws/amazon-ec2-instance-selector/pkg/selector"
78
"github.com/aws/aws-sdk-go/aws"
89
"github.com/aws/aws-sdk-go/aws/session"
@@ -27,10 +28,10 @@ func main() {
2728
LowerBound: 2,
2829
UpperBound: 4,
2930
}
30-
// Instantiate an int range filter to specify min and max memory in MiB
31-
memoryRange := selector.IntRangeFilter{
32-
LowerBound: 1024,
33-
UpperBound: 4096,
31+
// Instantiate a byte quantity range filter to specify min and max memory in GiB
32+
memoryRange := selector.ByteQuantityRangeFilter{
33+
LowerBound: bytequantity.FromGiB(2),
34+
UpperBound: bytequantity.FromGiB(4),
3435
}
3536
// Create a string for the CPU Architecture so that it can be passed as a pointer
3637
// when creating the Filter struct

cmd/main.go

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -116,11 +116,11 @@ Full docs can be found at github.com/aws/amazon-` + binName
116116
// Filter Flags - These will be grouped at the top of the help flags
117117

118118
cli.IntMinMaxRangeFlags(vcpus, cli.StringMe("c"), nil, "Number of vcpus available to the instance type.")
119-
cli.IntMinMaxRangeFlags(memory, cli.StringMe("m"), nil, "Amount of Memory available in MiB (Example: 4096)")
119+
cli.ByteQuantityMinMaxRangeFlags(memory, cli.StringMe("m"), nil, "Amount of Memory available (Example: 4 GiB)")
120120
cli.RatioFlag(vcpusToMemoryRatio, nil, nil, "The ratio of vcpus to memory in MiB. (Example: 1:2)")
121121
cli.StringFlag(cpuArchitecture, cli.StringMe("a"), nil, "CPU architecture [x86_64/amd64, i386, or arm64]", nil)
122122
cli.IntMinMaxRangeFlags(gpus, cli.StringMe("g"), nil, "Total Number of GPUs (Example: 4)")
123-
cli.IntMinMaxRangeFlags(gpuMemoryTotal, nil, nil, "Number of GPUs' total memory in MiB (Example: 4096)")
123+
cli.ByteQuantityMinMaxRangeFlags(gpuMemoryTotal, nil, nil, "Number of GPUs' total memory (Example: 4 GiB)")
124124
cli.StringFlag(placementGroupStrategy, nil, nil, "Placement group strategy: [cluster, partition, spread]", nil)
125125
cli.StringFlag(usageClass, cli.StringMe("u"), nil, "Usage class: [spot or on-demand]", nil)
126126
cli.StringFlag(rootDeviceType, nil, nil, "Supported root device types: [ebs or instance-store]", nil)
@@ -189,11 +189,11 @@ Full docs can be found at github.com/aws/amazon-` + binName
189189

190190
filters := selector.Filters{
191191
VCpusRange: cli.IntRangeMe(flags[vcpus]),
192-
MemoryRange: cli.IntRangeMe(flags[memory]),
192+
MemoryRange: cli.ByteQuantityRangeMe(flags[memory]),
193193
VCpusToMemoryRatio: cli.Float64Me(flags[vcpusToMemoryRatio]),
194194
CPUArchitecture: cli.StringMe(flags[cpuArchitecture]),
195195
GpusRange: cli.IntRangeMe(flags[gpus]),
196-
GpuMemoryRange: cli.IntRangeMe(flags[gpuMemoryTotal]),
196+
GpuMemoryRange: cli.ByteQuantityRangeMe(flags[gpuMemoryTotal]),
197197
PlacementGroupStrategy: cli.StringMe(flags[placementGroupStrategy]),
198198
UsageClass: cli.StringMe(flags[usageClass]),
199199
RootDeviceType: cli.StringMe(flags[rootDeviceType]),

pkg/bytequantity/bytequantity.go

Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
// Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License"). You may
4+
// not use this file except in compliance with the License. A copy of the
5+
// License is located at
6+
//
7+
// http://aws.amazon.com/apache2.0/
8+
//
9+
// or in the "license" file accompanying this file. This file is distributed
10+
// on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
11+
// express or implied. See the License for the specific language governing
12+
// permissions and limitations under the License.
13+
14+
package bytequantity
15+
16+
import (
17+
"fmt"
18+
"math"
19+
"regexp"
20+
"strconv"
21+
"strings"
22+
)
23+
24+
const (
25+
/// Examples: 1mb, 1 gb, 1.0tb, 1mib, 2g, 2.001 t
26+
byteQuantityRegex = `^([0-9]+\.?[0-9]{0,3})[ ]?(mi?b?|gi?b?|ti?b?)?$`
27+
mib = "MiB"
28+
gib = "GiB"
29+
tib = "TiB"
30+
gbConvert = 1 << 10
31+
tbConvert = gbConvert << 10
32+
maxGiB = math.MaxUint64 / gbConvert
33+
maxTiB = math.MaxUint64 / tbConvert
34+
)
35+
36+
// ByteQuantity is a data type representing a byte quantity
37+
type ByteQuantity struct {
38+
Quantity uint64
39+
}
40+
41+
// ParseToByteQuantity parses a string representation of a byte quantity to a ByteQuantity type.
42+
// A unit can be appended such as 16 GiB. If no unit is appended, GiB is assumed.
43+
func ParseToByteQuantity(byteQuantityStr string) (ByteQuantity, error) {
44+
bqRegexp := regexp.MustCompile(byteQuantityRegex)
45+
matches := bqRegexp.FindStringSubmatch(strings.ToLower(byteQuantityStr))
46+
if len(matches) < 2 {
47+
return ByteQuantity{}, fmt.Errorf("%s is not a valid byte quantity", byteQuantityStr)
48+
}
49+
50+
quantityStr := matches[1]
51+
unit := gib
52+
if len(matches) > 2 && matches[2] != "" {
53+
unit = matches[2]
54+
}
55+
quantity := uint64(0)
56+
switch strings.ToLower(string(unit[0])) {
57+
//mib
58+
case "m":
59+
inputDecSplit := strings.Split(quantityStr, ".")
60+
if len(inputDecSplit) == 2 {
61+
d, err := strconv.Atoi(inputDecSplit[1])
62+
if err != nil {
63+
return ByteQuantity{}, err
64+
}
65+
if d != 0 {
66+
return ByteQuantity{}, fmt.Errorf("cannot accept floating point MB value, only integers are accepted")
67+
}
68+
}
69+
// need error here so that this quantity doesn't bind in the local scope
70+
var err error
71+
quantity, err = strconv.ParseUint(inputDecSplit[0], 10, 64)
72+
if err != nil {
73+
return ByteQuantity{}, err
74+
}
75+
//gib
76+
case "g":
77+
quantityDec, err := strconv.ParseFloat(quantityStr, 10)
78+
if err != nil {
79+
return ByteQuantity{}, err
80+
}
81+
if quantityDec > maxGiB {
82+
return ByteQuantity{}, fmt.Errorf("error GiB value is too large")
83+
}
84+
quantity = uint64(quantityDec * gbConvert)
85+
//tib
86+
case "t":
87+
quantityDec, err := strconv.ParseFloat(quantityStr, 10)
88+
if err != nil {
89+
return ByteQuantity{}, err
90+
}
91+
if quantityDec > maxTiB {
92+
return ByteQuantity{}, fmt.Errorf("error TiB value is too large")
93+
}
94+
quantity = uint64(quantityDec * tbConvert)
95+
default:
96+
return ByteQuantity{}, fmt.Errorf("error unit %s is not supported", unit)
97+
}
98+
99+
return ByteQuantity{
100+
Quantity: quantity,
101+
}, nil
102+
}
103+
104+
// FromTiB returns a byte quantity of the passed in tebibytes quantity
105+
func FromTiB(tib uint64) ByteQuantity {
106+
return ByteQuantity{
107+
Quantity: tib * tbConvert,
108+
}
109+
}
110+
111+
// FromGiB returns a byte quantity of the passed in gibibytes quantity
112+
func FromGiB(gib uint64) ByteQuantity {
113+
return ByteQuantity{
114+
Quantity: gib * gbConvert,
115+
}
116+
}
117+
118+
// FromMiB returns a byte quantity of the passed in mebibytes quantity
119+
func FromMiB(mib uint64) ByteQuantity {
120+
return ByteQuantity{
121+
Quantity: mib,
122+
}
123+
}
124+
125+
// StringMiB returns a byte quantity in a mebibytes string representation
126+
func (bq ByteQuantity) StringMiB() string {
127+
return fmt.Sprintf("%.0f %s", bq.MiB(), mib)
128+
}
129+
130+
// StringGiB returns a byte quantity in a gibibytes string representation
131+
func (bq ByteQuantity) StringGiB() string {
132+
return fmt.Sprintf("%.3f %s", bq.GiB(), gib)
133+
}
134+
135+
// StringTiB returns a byte quantity in a tebibytes string representation
136+
func (bq ByteQuantity) StringTiB() string {
137+
return fmt.Sprintf("%.3f %s", bq.TiB(), tib)
138+
}
139+
140+
// MiB returns a byte quantity in mebibytes
141+
func (bq ByteQuantity) MiB() float64 {
142+
return float64(bq.Quantity)
143+
}
144+
145+
// GiB returns a byte quantity in gibibytes
146+
func (bq ByteQuantity) GiB() float64 {
147+
return float64(bq.Quantity) * 1 / gbConvert
148+
}
149+
150+
// TiB returns a byte quantity in tebibytes
151+
func (bq ByteQuantity) TiB() float64 {
152+
return float64(bq.Quantity) * 1 / tbConvert
153+
}

0 commit comments

Comments
 (0)