Tweaked vSwarm deployer by aryans1204 · Pull Request #604 · vhive-serverless/invitro

aryans1204 · 2025-02-10T11:56:33Z

Summary

A small summary of the requirements (in one/two sentences).

Implementation Notes ⚒️

Briefly outline the overall technical solution. If necessary, identify talking points where the reviewer's attention should be drawn to.
Goal of this PR is to add functionality in the loader to be able to execute vSwarm functions from a mapper_output.json file.

The added functionality pertains to:

Added parsing capabilities under pkg/trace/mapper_trace_parser.go to parse mapper_output.json files and extract vSwarm functions.
Added deployment capability under pkg/driver/deployment/knative.go to correctly deploy vSwarm functions, and a yamls.tar.gz tarball that contains the YAML files for all the vSwarm functions. Documentation about this tarball is embedded inside the tarball.

This extends upon the work done previously on the invoker, to be able to execute vSwarm functions. With the merge of this PR, vSwarm functions will be able to be executed from the loader, by specifying config options.

Added parsing, and deployment functionality to run vSwarm functions using Invitro loader, based on mapper output.

External Dependencies 🍀

Breaking API Changes ⚠️

Simply specify none (N/A) if not applicable.

leokondrashov

Please explain in details why and what are you doing in the PR description. The template is there for a reason.

I don't see any documentation in the changes as well. There should be something about the feature, file formats (you are introducing several).

pkg/common/trace_types.go

cmd/config_vswarm_trace.json

cmd/loader.go

pkg/trace/mapper_trace_parser.go

.github/workflows/e2e.yaml

leokondrashov

Please add docs on all files that you add as inputs to the execution. I have no idea how to get deploy_info and deployment yamls for a new set of benchmarks. What are the requirements for those benchmarks (use the relay from vSwarm, at least). Where is each file should be placed to be properly found by the loader.

cmd/loader.go

pkg/trace/test_data/test_deploy_info.json

pkg/trace/mapper_trace_parser.go

pkg/driver/deployment/knative.go

docs/generate_deploy_info_docs.md

workloads/container/.gitattributes

workloads/container/vSwarm_deploy_metadata.tar.gz

workloads/container/generate_all_yamls.py

docs/generate_deploy_info_docs.md

docs/loader.md

docs/generate_deploy_info_docs.md

pkg/trace/mapper_trace_parser.go

leokondrashov

Looks better. Please ask @cvetkovic and @JooyoungPark73 for review as well

docs/generate_deploy_info_docs.md

docs/loader.md

pkg/driver/deployment/knative.go

pkg/trace/mapper_trace_parser.go

aryans1204 · 2025-03-04T16:11:02Z

@cvetkovic do you have any other comments?

cvetkovic · 2025-03-06T13:06:36Z

@cvetkovic do you have any other comments?
No. @leokondrashov: Approve the PR if all good.

Signed-off-by: aryans1204 <arshar1204@gmail.com>

Signed-off-by: aryans1204 <arshar1204@gmail.com> Added tests to CI and formatted code Signed-off-by: aryans1204 <arshar1204@gmail.com> Fixing the E2E test for vSwarm loader Signed-off-by: aryans1204 <arshar1204@gmail.com> Extracted vSwarm mock server from standard workload Signed-off-by: aryans1204 <arshar1204@gmail.com>

Signed-off-by: aryans1204 <arshar1204@gmail.com> Fixed loader e2e tests Signed-off-by: aryans1204 <arshar1204@gmail.com>

Signed-off-by: aryans1204 <arshar1204@gmail.com> Beautified code according to Invitro standards Signed-off-by: aryans1204 <arshar1204@gmail.com> Added deployment of predeployment commands for vSwarm benchmarks Signed-off-by: aryans1204 <arshar1204@gmail.com> Removed erroneous E2E testfile Signed-off-by: aryans1204 <arshar1204@gmail.com> Added cleanup for kubectl pods Signed-off-by: aryans1204 <arshar1204@gmail.com> Fixed trace unit test Signed-off-by: aryans1204 <arshar1204@gmail.com> Changed function identifier to include hashOwner and hashApp Signed-off-by: aryans1204 <arshar1204@gmail.com> Added docs for deploy_info Signed-off-by: aryans1204 <arshar1204@gmail.com> Fixed spelling errors Signed-off-by: aryans1204 <arshar1204@gmail.com> Added docs for deploy info file generation Signed-off-by: aryans1204 <arshar1204@gmail.com> Created spelling fix for loader docs Signed-off-by: aryans1204 <arshar1204@gmail.com> Changed pre-deployment to Go command from bash script Signed-off-by: aryans1204 <arshar1204@gmail.com> Impoved documentation Signed-off-by: aryans1204 <arshar1204@gmail.com> Impoved documentation with spellcheck Signed-off-by: aryans1204 <arshar1204@gmail.com> Improved documentation Signed-off-by: aryans1204 <arshar1204@gmail.com> Fixed spellcheck in documentation Signed-off-by: aryans1204 <arshar1204@gmail.com> Updated docs Signed-off-by: aryans1204 <arshar1204@gmail.com> Changed log.Warn to log.Fatal Signed-off-by: aryans1204 <arshar1204@gmail.com>

aryans1204 requested review from cvetkovic and leokondrashov and removed request for leokondrashov February 10, 2025 11:57

leokondrashov reviewed Feb 10, 2025

View reviewed changes

pkg/common/trace_types.go Outdated Show resolved Hide resolved

cmd/config_vswarm_trace.json Show resolved Hide resolved

aryans1204 requested a review from leokondrashov February 11, 2025 09:15

cvetkovic requested changes Feb 11, 2025

View reviewed changes

cmd/loader.go Outdated Show resolved Hide resolved

pkg/trace/mapper_trace_parser.go Show resolved Hide resolved

pkg/trace/mapper_trace_parser.go Show resolved Hide resolved

pkg/trace/mapper_trace_parser.go Show resolved Hide resolved

aryans1204 requested a review from cvetkovic February 12, 2025 13:57

cvetkovic previously approved these changes Feb 12, 2025

View reviewed changes

leokondrashov reviewed Feb 13, 2025

View reviewed changes

pkg/trace/mapper_trace_parser.go Outdated Show resolved Hide resolved

.github/workflows/e2e.yaml Outdated Show resolved Hide resolved

aryans1204 dismissed cvetkovic’s stale review via 4adbd4e February 13, 2025 08:00

aryans1204 requested a review from leokondrashov February 13, 2025 09:06

leokondrashov reviewed Feb 14, 2025

View reviewed changes

cmd/loader.go Outdated Show resolved Hide resolved

pkg/trace/test_data/test_deploy_info.json Show resolved Hide resolved

pkg/trace/mapper_trace_parser.go Outdated Show resolved Hide resolved

aryans1204 requested a review from leokondrashov February 18, 2025 07:04

aryans1204 force-pushed the new-vswarm-deployer branch 2 times, most recently from 5c945cf to b8d098b Compare February 24, 2025 06:18

aryans1204 requested a review from JooyoungPark73 February 24, 2025 09:08

leokondrashov reviewed Feb 25, 2025

View reviewed changes

aryans1204 force-pushed the new-vswarm-deployer branch from f011f41 to 601e0a6 Compare February 26, 2025 09:28

aryans1204 requested a review from leokondrashov February 26, 2025 09:38

leokondrashov reviewed Feb 26, 2025

View reviewed changes

aryans1204 force-pushed the new-vswarm-deployer branch from ca58c49 to 6befe01 Compare February 27, 2025 03:30

aryans1204 requested a review from leokondrashov February 27, 2025 03:44

leokondrashov reviewed Feb 27, 2025

View reviewed changes

docs/generate_deploy_info_docs.md Outdated Show resolved Hide resolved

docs/generate_deploy_info_docs.md Outdated Show resolved Hide resolved

docs/loader.md Outdated Show resolved Hide resolved

docs/loader.md Outdated Show resolved Hide resolved

aryans1204 requested a review from cvetkovic February 27, 2025 16:18

cvetkovic reviewed Feb 28, 2025

View reviewed changes

pkg/driver/deployment/knative.go Show resolved Hide resolved

pkg/trace/mapper_trace_parser.go Show resolved Hide resolved

aryans1204 force-pushed the new-vswarm-deployer branch from b0faf74 to 403ab19 Compare March 1, 2025 16:28

aryans1204 requested review from cvetkovic and leokondrashov March 1, 2025 17:29

aryans1204 force-pushed the new-vswarm-deployer branch from 403ab19 to a231f5b Compare March 4, 2025 06:05

aryans1204 force-pushed the new-vswarm-deployer branch from a231f5b to d5ca597 Compare March 4, 2025 16:26

leokondrashov previously approved these changes Mar 7, 2025

View reviewed changes

aryans1204 added 6 commits March 7, 2025 02:07

Created parser for mapper output of vSwarm functions

1a35a6b

Signed-off-by: aryans1204 <arshar1204@gmail.com>

Added deployment for vSwarm functions, end to end pipeline working

446683a

Signed-off-by: aryans1204 <arshar1204@gmail.com>

Created vSwarm YAML manifest tarball

13a66a4

Signed-off-by: aryans1204 <arshar1204@gmail.com>

Changed JSON trace parser to include struct instead of interface

eb17a29

Signed-off-by: aryans1204 <arshar1204@gmail.com> Fixed loader e2e tests Signed-off-by: aryans1204 <arshar1204@gmail.com>

aryans1204 dismissed leokondrashov’s stale review via 7a4502d March 7, 2025 08:15

aryans1204 force-pushed the new-vswarm-deployer branch from d5ca597 to 7a4502d Compare March 7, 2025 08:15

leokondrashov approved these changes Mar 7, 2025

View reviewed changes

leokondrashov merged commit 7686583 into main Mar 7, 2025
15 checks passed

leokondrashov deleted the new-vswarm-deployer branch March 7, 2025 08:43

Conversation

aryans1204 commented Feb 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Implementation Notes ⚒️

External Dependencies 🍀

Breaking API Changes ⚠️

Uh oh!

leokondrashov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

leokondrashov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

leokondrashov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aryans1204 commented Mar 4, 2025

Uh oh!

cvetkovic commented Mar 6, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

aryans1204 commented Feb 10, 2025 •

edited

Loading