Skip to content

Refine benchmark script logic #14

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from
Open
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions calibration_boards/objdetect_benchmark.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,10 +53,10 @@ def get_norm(gold_corners, corners, type_dist):


def get_max_error(accuracy_threshold, type_dist):
if type_dist is TypeNorm.l1 or TypeNorm.l2 or TypeNorm.l3 or TypeNorm.l_inf:
return accuracy_threshold
if type_dist in (TypeNorm.l1, TypeNorm.l2, TypeNorm.l_inf):
return accuracy_threshold
if type_dist is TypeNorm.intersection_over_union:
return 1
return 1.0
raise TypeError("this TypeNorm isn't supported")


Expand All @@ -77,15 +77,15 @@ def get_synthetic_rt(yaw, pitch, distance):
def get_coord(num_rows, num_cols, start_x=0, start_y=0):
i, j = np.ogrid[:num_rows, :num_cols]
v = np.empty((num_rows, num_cols, 2), dtype=np.float32)
v[..., 0] = j + start_y
v[..., 1] = i + start_x
v[..., 0] = j + start_x
v[..., 1] = i + start_y
v.shape = (1, -1, 2)
return v
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function is not used in the benchmark at all and may be removed.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I will remove it.



class TransformObject:
def __init__(self):
self.name = "none"
self.name = ""

def transform_image(self, image):
return image
Expand Down Expand Up @@ -825,7 +825,7 @@ def main():
parser.add_argument("--board_y", help="input board y size", default="6", action="store", dest="board_y", type=int)
parser.add_argument("--rel_center_x", help="the relative x-axis location of the center of the board in the image",
default=".5", action="store", dest="rel_center_x", type=float)
parser.add_argument("--rel_center_y", help="the relative x-axis location of the center of the board in the image",
parser.add_argument("--rel_center_y", help="the relative y-axis location of the center of the board in the image",
default=".5", action="store", dest="rel_center_y", type=float)
parser.add_argument("--metric", help="Metric for distance between result and gold", default="l_inf", action="store",
dest="metric", choices=['l1', 'l2', 'l_inf', 'intersection_over_union'], type=str)
Expand Down Expand Up @@ -886,12 +886,12 @@ def main():

folder_name = "report_" + get_time()
configuration = args.configuration
if configuration == "generate" or configuration == "generate_run":
if configuration in ("generate", "generate_run"):
generate_dataset(args, synthetic_object)
if configuration == "generate":
return
return
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It changes logic. The original "generate" just saves test images and exit. "generate_run" generates, saves and runs all benchmark steps.

elif configuration == "show":
distances1 = read_distances(dataset_path+'/'+args.d1)
file = args.d1 if args.d1.endswith('.json') else args.d1 + '.json'
distances1 = read_distances(os.path.join(dataset_path, file))
distances3 = distances1
if args.d2 != '':
distances2 = read_distances(dataset_path + '/' + args.d2)
Expand Down