Skip to content

fix the can VM fit on Host check, added Unit tests for the SimHost #306

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
vlogic wants to merge 33 commits into
base: master
Choose a base branch
from
Open

fix the can VM fit on Host check, added Unit tests for the SimHost #306

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
97eab8b
fix the can VM fit on Host check, added Unit tests for the SimHost
vlogic Feb 17, 2025
e56f2e4
fix format violations
vlogic Feb 17, 2025
3cd1e80
Update libs.versions.toml
vlogic Feb 17, 2025
003fa97
fix format violations
vlogic Feb 17, 2025
adc88cd
fix code coverage reporting
vlogic Feb 17, 2025
6047a44
Fixed bugs (#304)
DanteNiewenhuis Feb 17, 2025
d5dbbad
revert fix code coverage reporting
vlogic Feb 17, 2025
acdb097
test new token
vlogic Feb 17, 2025
94d2f04
test with new permissions
vlogic Feb 17, 2025
5bf9962
test with new permissions
vlogic Feb 17, 2025
651a10c
test with new permissions
vlogic Feb 17, 2025
6e0266f
update gradle github action package
vlogic Feb 17, 2025
6e631f0
update gradle github action package
vlogic Feb 17, 2025
44bb24c
revert update gradle github action package
vlogic Feb 17, 2025
c961aa0
revert exedental java upgrade
vlogic Feb 17, 2025
43b8440
run coverage in verbose mode
vlogic Feb 18, 2025
9b1def0
run coverage in verbose mode and enable cache overwrite
vlogic Feb 18, 2025
4867ac4
revert cache overwrite
vlogic Feb 18, 2025
772e640
test only upload cov report from ubuntu-22.04
vlogic Feb 18, 2025
f41c1ff
test old codecov/codecov-action@v5.0.4
vlogic Feb 18, 2025
23840f6
test old codecov/codecov-action@v5.2.0
vlogic Feb 18, 2025
fa29066
test cleanup before build
vlogic Feb 18, 2025
78eaa42
test cleanup before build
vlogic Feb 18, 2025
dcbb96c
test cleanup after build
vlogic Feb 18, 2025
aaff22a
use env var for CODECOV_TOKEN
vlogic Feb 18, 2025
6300226
diable search and dont use codecov token
vlogic Feb 18, 2025
865ab8a
remove verbosee
vlogic Feb 18, 2025
535db80
search for coverage reeports
vlogic Feb 18, 2025
ffbf127
add coverasge file
vlogic Feb 18, 2025
39c6701
test files paths
vlogic Feb 18, 2025
3ea24ec
test files paths
vlogic Feb 18, 2025
5723ab8
remove test files paths
vlogic Feb 18, 2025
f0a9168
Added the option to select which files to export. (#307)
DanteNiewenhuis Feb 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,10 +61,10 @@ jobs:
./**/build/distributions/**/*.zip
retention-days: 5
- name: Upload code coverage
if: matrix.os == 'ubuntu-22.04'
uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./build/reports/jacoco/codeCoverageReport/codeCoverageReport.xml
files: /home/runner/work/opendc/opendc/build/reports/jacoco/codeCoverageReport/codeCoverageReport.xml
build-docker:
name: Build Docker Images
runs-on: ubuntu-22.04
Expand Down
3 changes: 2 additions & 1 deletion ...c-compute-simulator/src/main/java/org/opendc/compute/simulator/service/ServiceFlavor.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,8 @@ public final class ServiceFlavor implements Flavor {
private final long memorySize;
private final Map<String, ?> meta;

ServiceFlavor(ComputeService service, UUID uid, String name, int coreCount, long memorySize, Map<String, ?> meta) {
public ServiceFlavor(
ComputeService service, UUID uid, String name, int coreCount, long memorySize, Map<String, ?> meta) {
this.service = service;
this.uid = uid;
this.name = name;
Expand Down
2 changes: 1 addition & 1 deletion ...ndc-compute-simulator/src/main/java/org/opendc/compute/simulator/service/ServiceTask.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ public class ServiceTask {

private int numFailures = 0;

ServiceTask(
public ServiceTask(
ComputeService service,
UUID uid,
String name,
Expand Down
45 changes: 36 additions & 9 deletions ...ute/opendc-compute-simulator/src/main/kotlin/org/opendc/compute/simulator/host/SimHost.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,15 +42,18 @@ import java.time.Instant
import java.time.InstantSource

/**
* A [SimHost] implementation that simulates virtual machines on a physical machine.
* The `SimHost` class represents a simulated host that operates as a hypervisor for running virtualized tasks.
* It manages its state, running tasks, associated virtual machines, and system metrics, while providing host
* lifecycle management and resource usage computation.
*
* @param name The name of the host.
* @param clock The (virtual) clock used to track time.
* @param graph The Flow Graph that the Host is part of
* @param machineModel The static model of the host
* @param cpuPowerModel The power model of the host
* @param powerDistributor The power distributor to which the host is connected
* @constructor Create empty Sim host
* @constructor Initializes the SimHost with required parameters.
* @param name The unique name of the simulated host.
* @param clusterName The cluster name to which this host belongs.
* @param clock The clock source to simulate time within the system.
* @param graph The data-flow graph used by the simulation for task operations.
* @param machineModel The hardware model attributes of the host.
* @param cpuPowerModel The CPU power characteristics of the host.
* @param powerDistributor The power distributor managing system energy allocations.
*/
public class SimHost(
private val name: String,
Expand Down Expand Up @@ -191,8 +194,32 @@ public class SimHost(
return this.guests
}

/**
* Calculates the total memory used by the currently running tasks on the host.
*
* Iterates through the tasks mapped to guests in `taskToGuestMap`. For tasks that are in the
* `TaskState.RUNNING` state, their memory consumption is summed up.
*
* @return Total memory used by tasks currently in the RUNNING state, in bytes.
*/
private fun usedMemoryByRunningTasks(): Long {
var usedMemory: Long = 0
for (vm in this.taskToGuestMap) {
if (vm.value.state == TaskState.RUNNING) {
usedMemory += vm.key.flavor.memorySize
}
}
return usedMemory
}

/**
* Determines if the given task can fit on this host based on its resource requirements.
*
* @param task The task to be checked. It includes the resource requirements such as memory size and core count.
* @return True if the task can fit on this host considering available memory, CPU cores, and machine model compatibility; false otherwise.
*/
public fun canFit(task: ServiceTask): Boolean {
val sufficientMemory = model.memoryCapacity >= task.flavor.memorySize
val sufficientMemory = (model.memoryCapacity - this.usedMemoryByRunningTasks()) >= task.flavor.memorySize
val enoughCpus = model.coreCount >= task.flavor.coreCount
val canFit = simMachine!!.canFit(task.flavor.toMachineModel())

Expand Down
10 changes: 10 additions & 0 deletions ...rc/main/kotlin/org/opendc/compute/simulator/provisioner/ComputeMonitorProvisioningStep.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ package org.opendc.compute.simulator.provisioner
import org.opendc.compute.simulator.service.ComputeService
import org.opendc.compute.simulator.telemetry.ComputeMetricReader
import org.opendc.compute.simulator.telemetry.ComputeMonitor
import org.opendc.compute.simulator.telemetry.OutputFiles
import java.time.Duration

/**
Expand All @@ -36,6 +37,14 @@ public class ComputeMonitorProvisioningStep(
private val monitor: ComputeMonitor,
private val exportInterval: Duration,
private val startTime: Duration = Duration.ofMillis(0),
private val filesToExport: Map<OutputFiles, Boolean> =
mapOf(
OutputFiles.HOST to true,
OutputFiles.TASK to true,
OutputFiles.SERVICE to true,
OutputFiles.POWER_SOURCE to true,
OutputFiles.BATTERY to true,
),
) : ProvisioningStep {
override fun apply(ctx: ProvisioningContext): AutoCloseable {
val service =
Expand All @@ -49,6 +58,7 @@ public class ComputeMonitorProvisioningStep(
monitor,
exportInterval,
startTime,
filesToExport,
)
return metricReader
}
Expand Down
11 changes: 10 additions & 1 deletion ...ompute-simulator/src/main/kotlin/org/opendc/compute/simulator/provisioner/ComputeSteps.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ package org.opendc.compute.simulator.provisioner

import org.opendc.compute.simulator.scheduler.ComputeScheduler
import org.opendc.compute.simulator.telemetry.ComputeMonitor
import org.opendc.compute.simulator.telemetry.OutputFiles
import org.opendc.compute.topology.specs.ClusterSpec
import org.opendc.compute.topology.specs.HostSpec
import java.time.Duration
Expand Down Expand Up @@ -59,8 +60,16 @@ public fun registerComputeMonitor(
monitor: ComputeMonitor,
exportInterval: Duration = Duration.ofMinutes(5),
startTime: Duration = Duration.ofMillis(0),
filesToExport: Map<OutputFiles, Boolean> =
mapOf(
OutputFiles.HOST to true,
OutputFiles.TASK to true,
OutputFiles.SERVICE to true,
OutputFiles.POWER_SOURCE to true,
OutputFiles.BATTERY to true,
),
): ProvisioningStep {
return ComputeMonitorProvisioningStep(serviceDomain, monitor, exportInterval, startTime)
return ComputeMonitorProvisioningStep(serviceDomain, monitor, exportInterval, startTime, filesToExport)
}

/**
Expand Down
116 changes: 67 additions & 49 deletions ...e-simulator/src/main/kotlin/org/opendc/compute/simulator/telemetry/ComputeMetricReader.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,14 @@ public class ComputeMetricReader(
private val monitor: ComputeMonitor,
private val exportInterval: Duration = Duration.ofMinutes(5),
private val startTime: Duration = Duration.ofMillis(0),
private val toMonitor: Map<OutputFiles, Boolean> =
mapOf(
OutputFiles.HOST to true,
OutputFiles.TASK to true,
OutputFiles.POWER_SOURCE to true,
OutputFiles.BATTERY to true,
OutputFiles.SERVICE to true,
),
) : AutoCloseable {
private val logger = KotlinLogging.logger {}
private val scope = CoroutineScope(dispatcher.asCoroutineDispatcher())
Expand Down Expand Up @@ -116,68 +124,78 @@ public class ComputeMetricReader(
try {
val now = this.clock.instant()

for (host in this.service.hosts) {
val reader =
this.hostTableReaders.computeIfAbsent(host) {
HostTableReaderImpl(
it,
startTime,
)
}
reader.record(now)
this.monitor.record(reader.copy())
reader.reset()
if (toMonitor[OutputFiles.HOST] == true) {
for (host in this.service.hosts) {
val reader =
this.hostTableReaders.computeIfAbsent(host) {
HostTableReaderImpl(
it,
startTime,
)
}
reader.record(now)
this.monitor.record(reader.copy())
reader.reset()
}
}

for (task in this.service.tasks) {
val reader =
this.taskTableReaders.computeIfAbsent(task) {
TaskTableReaderImpl(
service,
it,
startTime,
)
}
reader.record(now)
this.monitor.record(reader.copy())
reader.reset()
if (toMonitor[OutputFiles.TASK] == true) {
for (task in this.service.tasks) {
val reader =
this.taskTableReaders.computeIfAbsent(task) {
TaskTableReaderImpl(
service,
it,
startTime,
)
}
reader.record(now)
this.monitor.record(reader.copy())
reader.reset()
}
}

for (task in this.service.tasksToRemove) {
task.delete()
}
this.service.clearTasksToRemove()

for (simPowerSource in this.service.powerSources) {
val reader =
this.powerSourceTableReaders.computeIfAbsent(simPowerSource) {
PowerSourceTableReaderImpl(
it,
startTime,
)
}

reader.record(now)
this.monitor.record(reader.copy())
reader.reset()
if (toMonitor[OutputFiles.POWER_SOURCE] == true) {
for (simPowerSource in this.service.powerSources) {
val reader =
this.powerSourceTableReaders.computeIfAbsent(simPowerSource) {
PowerSourceTableReaderImpl(
it,
startTime,
)
}

reader.record(now)
this.monitor.record(reader.copy())
reader.reset()
}
}

for (simBattery in this.service.batteries) {
val reader =
this.batteryTableReaders.computeIfAbsent(simBattery) {
BatteryTableReaderImpl(
it,
startTime,
)
}

reader.record(now)
this.monitor.record(reader.copy())
reader.reset()
if (toMonitor[OutputFiles.BATTERY] == true) {
for (simBattery in this.service.batteries) {
val reader =
this.batteryTableReaders.computeIfAbsent(simBattery) {
BatteryTableReaderImpl(
it,
startTime,
)
}

reader.record(now)
this.monitor.record(reader.copy())
reader.reset()
}
}

this.serviceTableReader.record(now)
monitor.record(this.serviceTableReader.copy())
if (toMonitor[OutputFiles.SERVICE] == true) {
this.serviceTableReader.record(now)
monitor.record(this.serviceTableReader.copy())
}

if (loggCounter >= 100) {
var loggString = "\n\t\t\t\t\tMetrics after ${now.toEpochMilli() / 1000 / 60 / 60} hours:\n"
Expand Down
44 changes: 44 additions & 0 deletions ...c-compute-simulator/src/main/kotlin/org/opendc/compute/simulator/telemetry/OutputFiles.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
/*
* Copyright (c) 2025 AtLarge Research
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/

package org.opendc.compute.simulator.telemetry

import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable

@Serializable
public enum class OutputFiles {
@SerialName("host")
HOST,

@SerialName("task")
TASK,

@SerialName("powerSource")
POWER_SOURCE,

@SerialName("battery")
BATTERY,

@SerialName("service")
SERVICE,
}
Loading