Skip to content

Support scraping multiple controllers. #27

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sebbov wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Support scraping multiple controllers. #27

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e15e190
Support scraping multiple controllers.
sebbov Jan 8, 2025
c4e9ecd
[review] Doc support for multiple controllers.
sebbov Feb 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,23 @@ Prometheus exporter for Areca RAID cards. Exporter depends on Areca CLI being pr

## Features

- Provides metrics for the Areca RAID card to be scraped by Prometheus.
- Provides metrics for the Areca RAID cards to be scraped by Prometheus.
- Supports the following metrics:
- `areca_up`: '0' if a scrape of the Areca CLI was successful, '1' otherwise.
- `areca_sys_info`: Constant metric with a value of 1 labeled with information about the Areca controller.
- `areca_sys_info`: Constant metric with a value of 1 labeled with information about Areca controllers.
- `areca_raid_set_state`: Areca RAID set state, where 0 represents normal and 1 represents degraded.
- `areca_disk_info`: Constant metric with value 1 labeled with info about all physical disks attached to the Areca controller.
- `areca_disk_info`: Constant metric with value 1 labeled with info about all physical disks attached to Areca controllers.
- `areca_disk_state`: Areca controller metric for disk state, 0 for normal, 1 for error
- `areca_disk_media_errors`: Metric for media errors of all physical disks attached to the Areca controller.
- `areca_disk_media_errors`: Metric for media errors of all physical disks attached to Areca controllers.
Comment on lines +7 to +14
Copy link
Owner

@vilhelmprytz vilhelmprytz Feb 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Maybe we could emphasize that the exporter supports systems with multiple controllers as well?

Copy link
Author

@sebbov sebbov Feb 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

- Supports systems with multiple controllers.

## Config options

| Option | Description | Default |
| -------------------- | --------------------------- | ------------- |
| `--collect-interval` | How often to poll Areca CLI | `5s` |
| `--cli-path` | Path to Areca CLI binary | `areca.cli64` |
| Option | Description | Default |
| -------------------- | --------------------------------- | ------------- |
| `--collect-interval` | How often to poll each controller | `5s` |
| `--cli-path` | Path to Areca CLI binary | `areca.cli64` |
| `--controllers` | How many controllers to scrape | `1` |
Copy link
Owner

@vilhelmprytz vilhelmprytz Feb 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a little bit out of scope, but would it be possible to auto-detect the number of controllers? For example, by "trying" different n = 1, 2, 3, ... etc in numctrl=<n> and see when it fails?

Copy link
Author

@sebbov sebbov Feb 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if you saw the pr description covering that? I initially implemented exactly what you suggest but I felt it was fragile for the reasons given there. I then considered the current behavior of the exporter: It is run under the implicit assumption that there is a controller at index 1, and report errors if failures are encountered (i.e., it doesn't auto-discover there are no controllers and report success, doing nothing). So, making that intent explicit and supporting more than 1, reporting on failures for controllers which aren't actually found (rather than ignoring a difference between intent and actual), seemed to both avoid that fragility and match the current spirit well. I understand it's slightly less functional, but I felt it was a good tradeoff.


## Prerequisites

Expand Down
140 changes: 80 additions & 60 deletions areca_exporter.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,13 +30,24 @@ const (
default_port = 9423
)

func runArecaCli(cmd string) ([]byte, error) {
type controller struct {
index int
arecaSysInfoUp, arecaRsfInfoUp, arecaDiskInfoUp prometheus.Gauge
}

func (ctrl controller) newLabels() map[string]string {
return map[string]string{
"controller_index": strconv.Itoa(ctrl.index),
}
}

func (ctrl controller) runArecaCli(cmd string) ([]byte, error) {
var cancel context.CancelFunc
var ctx context.Context
ctx, cancel = context.WithTimeout(context.Background(), time.Duration(60)*time.Second)
defer cancel()

out, err := exec.CommandContext(ctx, *cliPath, cmd).Output()
out, err := exec.CommandContext(ctx, *cliPath, fmt.Sprintf("curctrl=%d", ctrl.index), cmd).Output()

if err != nil {
level.Error(logger).Log("err", err, "msg", out)
Expand All @@ -45,25 +56,25 @@ func runArecaCli(cmd string) ([]byte, error) {
return out, err
}

func getSysInfo() prometheus.Labels {
out, cmd_err := runArecaCli("sys info")
func (ctrl controller) getSysInfo() prometheus.Labels {
out, cmd_err := ctrl.runArecaCli("sys info")

if cmd_err != nil {
arecaSysInfoUp.Set(1)
ctrl.arecaSysInfoUp.Set(1)
return nil
}

defer func() {
if panicInfo := recover(); panicInfo != nil {
level.Error(logger).Log("err", panicInfo, "msg", debug.Stack())
arecaSysInfoUp.Set(1)
ctrl.arecaSysInfoUp.Set(1)
}
}()

// split by newline, look for ": " and split by that
// then trim the space from the key and value
// then add to map
m := make(map[string]string)
m := ctrl.newLabels()
for _, line := range bytes.Split(out, []byte("\n")) {
if bytes.Contains(line, []byte(": ")) {
kv := bytes.Split(line, []byte(": "))
Expand All @@ -83,23 +94,23 @@ func getSysInfo() prometheus.Labels {
}
}

arecaDiskInfoUp.Set(0)
ctrl.arecaDiskInfoUp.Set(0)

return prometheus.Labels(m)
}

func getRaidSetInfo() []map[string]string {
out, cmd_err := runArecaCli("rsf info")
func (ctrl controller) getRaidSetInfo() []map[string]string {
out, cmd_err := ctrl.runArecaCli("rsf info")

if cmd_err != nil {
arecaRsfInfoUp.Set(1)
ctrl.arecaRsfInfoUp.Set(1)
return nil
}

defer func() {
if panicInfo := recover(); panicInfo != nil {
level.Error(logger).Log("err", panicInfo, "msg", debug.Stack())
arecaRsfInfoUp.Set(1)
ctrl.arecaRsfInfoUp.Set(1)
}
}()

Expand Down Expand Up @@ -140,7 +151,7 @@ func getRaidSetInfo() []map[string]string {
}

// add to hashmap
m := make(map[string]string)
m := ctrl.newLabels()

for i, key := range headerKeys {
if key == "name" {
Expand All @@ -153,23 +164,23 @@ func getRaidSetInfo() []map[string]string {
raidSets = append(raidSets, m)
}

arecaRsfInfoUp.Set(0)
ctrl.arecaRsfInfoUp.Set(0)

return raidSets
}

func getDiskInfo() []map[string]string {
out, cmd_err := runArecaCli("disk info")
func (ctrl controller) getDiskInfo() []map[string]string {
out, cmd_err := ctrl.runArecaCli("disk info")

if cmd_err != nil {
arecaDiskInfoUp.Set(1)
ctrl.arecaDiskInfoUp.Set(1)
return nil
}

defer func() {
if panicInfo := recover(); panicInfo != nil {
level.Error(logger).Log("err", panicInfo, "msg", debug.Stack())
arecaDiskInfoUp.Set(1)
ctrl.arecaDiskInfoUp.Set(1)
}
}()

Expand Down Expand Up @@ -210,7 +221,7 @@ func getDiskInfo() []map[string]string {
}

// add to hashmap
m := make(map[string]string)
m := ctrl.newLabels()

for i, key := range headerKeys {
m[key] = disk[i]
Expand All @@ -219,18 +230,18 @@ func getDiskInfo() []map[string]string {
disks = append(disks, m)
}

arecaDiskInfoUp.Set(0)
ctrl.arecaDiskInfoUp.Set(0)

return disks
}

func getDetailedDiskInfo(disk map[string]string) map[string]string {
func (ctrl controller) getDetailedDiskInfo(disk map[string]string) map[string]string {
if disk["modelname"] == "N.A." {
return nil
}

// get detailed disk info
out, cmd_err := runArecaCli(fmt.Sprintf("disk info drv=%s", disk["num"]))
out, cmd_err := ctrl.runArecaCli(fmt.Sprintf("disk info drv=%s", disk["num"]))

if cmd_err != nil {
return nil
Expand All @@ -239,11 +250,11 @@ func getDetailedDiskInfo(disk map[string]string) map[string]string {
defer func() {
if panicInfo := recover(); panicInfo != nil {
level.Error(logger).Log("err", panicInfo, "msg", debug.Stack())
arecaDiskInfoUp.Set(1)
ctrl.arecaDiskInfoUp.Set(1)
}
}()

m := make(map[string]string)
m := ctrl.newLabels()
m["num"] = disk["num"]

// Split output into keys (column 1) and values (column 2)
Expand Down Expand Up @@ -310,17 +321,18 @@ func regRsfMetric(rsf_info map[string]string) prometheus.Gauge {
return raidSet
}

func recordMetrics() {
func (ctrl controller) recordMetrics() {
// record sys info initially
var arecaSysInfo = promauto.NewGauge(prometheus.GaugeOpts{
Name: "areca_sys_info",
Help: "Constant metric with value 1 labeled with info about Areca controller.",
ConstLabels: getSysInfo(),
})
if labels := ctrl.getSysInfo(); labels != nil {
promauto.NewGauge(prometheus.GaugeOpts{
Name: "areca_sys_info",
Help: "Constant metric with value 1 labeled with info about Areca controller.",
ConstLabels: labels,
}).Set(1)
}

arecaSysInfo.Set(1)
arecaRsfInfoUp.Set(0)
arecaDiskInfoUp.Set(0)
ctrl.arecaRsfInfoUp.Set(0)
ctrl.arecaDiskInfoUp.Set(0)

// create new gauge for each raid set, and each disk
var raidSetGauges []prometheus.Gauge
Expand All @@ -332,10 +344,10 @@ func recordMetrics() {
go func() {
for {
// get new raid set info
rsf_info := getRaidSetInfo()
rsf_info := ctrl.getRaidSetInfo()

// get new disk info
disk_info := getDiskInfo()
disk_info := ctrl.getDiskInfo()

// if same amount of raid sets, then just update the labels if changed
if len(raidSetGauges) == len(rsf_info) {
Expand Down Expand Up @@ -379,7 +391,7 @@ func recordMetrics() {
diskGauges = append(diskGauges, disk)

// get media errors and state per disk and create metrics
if detailed_disk_info := getDetailedDiskInfo(m); detailed_disk_info != nil {
if detailed_disk_info := ctrl.getDetailedDiskInfo(m); detailed_disk_info != nil {
mediaErrorLabels, mediaErrorValue := getMediaErrors(detailed_disk_info)

// ignore disks with no media error value, i.e. on very old Areca controllers
Expand All @@ -404,7 +416,6 @@ func recordMetrics() {
diskStateGauges = append(diskStateGauges, diskStateGauge)
}
}

time.Sleep(*collectInterval)
}
}()
Expand All @@ -413,31 +424,12 @@ func recordMetrics() {

var (
logger = promlog.New(&promlog.Config{})
collectInterval = kingpin.Flag("collect-interval", "How often to poll Areca CLI").Default("5s").Duration()
collectInterval = kingpin.Flag("collect-interval", "How often to poll each controller").Default("5s").Duration()
cliPath = kingpin.Flag("cli-path", "Path to the Areca CLI binary").Default("areca.cli64").String()
controllers = kingpin.Flag("controllers", "How many controllers to scrape").Default("1").Int()

arecaSysInfoUp = promauto.NewGauge(prometheus.GaugeOpts{
Name: "areca_up",
Help: "'0' if a scrape of the Areca CLI was successful, '1' otherwise.",
ConstLabels: prometheus.Labels{
"collector": "sys_info",
},
})
arecaRsfInfoUp = promauto.NewGauge(prometheus.GaugeOpts{
Name: "areca_up",
Help: "'0' if a scrape of the Areca CLI was successful, '1' otherwise.",
ConstLabels: prometheus.Labels{
"collector": "rsf_info",
},
})
arecaDiskInfoUp = promauto.NewGauge(prometheus.GaugeOpts{
Name: "areca_up",
Help: "'0' if a scrape of the Areca CLI was successful, '1' otherwise.",
ConstLabels: prometheus.Labels{
"collector": "disk_info",
},
})
diskLabels = []string{
"controller_index",
"device_location",
"device_type",
"disk_capacity",
Expand All @@ -460,7 +452,35 @@ func main() {
log.Fatalf("could not register version collector: %v", err)
}

recordMetrics()
for i := 1; i <= *controllers; i++ {
controller{
index: i,
arecaSysInfoUp: promauto.NewGauge(prometheus.GaugeOpts{
Name: "areca_up",
Help: "'0' if a scrape of the Areca CLI was successful, '1' otherwise.",
ConstLabels: prometheus.Labels{
"collector": "sys_info",
"controller_index": strconv.Itoa(i),
},
}),
arecaRsfInfoUp: promauto.NewGauge(prometheus.GaugeOpts{
Name: "areca_up",
Help: "'0' if a scrape of the Areca CLI was successful, '1' otherwise.",
ConstLabels: prometheus.Labels{
"collector": "rsf_info",
"controller_index": strconv.Itoa(i),
},
}),
arecaDiskInfoUp: promauto.NewGauge(prometheus.GaugeOpts{
Name: "areca_up",
Help: "'0' if a scrape of the Areca CLI was successful, '1' otherwise.",
ConstLabels: prometheus.Labels{
"collector": "disk_info",
"controller_index": strconv.Itoa(i),
},
}),
}.recordMetrics()
}

level.Info(logger).Log("msg", "Starting areca_exporter", "version", version.Info())
level.Info(logger).Log("msg", "Build context", "build_context", version.BuildContext())
Expand Down