-
Notifications
You must be signed in to change notification settings - Fork 162
feat: dictionary provider for Strings and Integrals #975
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 1 commit
4884648
986ede1
ac12f41
66cc9b4
770e8f8
bc4a764
60a488b
4461462
84240e5
e9f25e2
84707f2
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| /* | ||
| * Copyright 2024 Code Intelligence GmbH | ||
| * | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| package com.code_intelligence.jazzer.mutation.annotation; | ||
|
|
||
| import static com.code_intelligence.jazzer.mutation.utils.PropertyConstraint.RECURSIVE; | ||
| import static java.lang.annotation.ElementType.TYPE_USE; | ||
| import static java.lang.annotation.RetentionPolicy.RUNTIME; | ||
|
|
||
| import com.code_intelligence.jazzer.mutation.utils.IgnoreRecursiveConflicts; | ||
| import com.code_intelligence.jazzer.mutation.utils.PropertyConstraint; | ||
| import java.lang.annotation.ElementType; | ||
| import java.lang.annotation.Retention; | ||
| import java.lang.annotation.Target; | ||
|
|
||
| /** | ||
| * Provides dictionary values to user-selected mutator types. Currently supported mutators are: | ||
| * | ||
| * <ul> | ||
| * <li>String mutator | ||
| * <li>Integral mutators (byte, short, int, long) | ||
| * </ul> | ||
| * | ||
| * <p>This annotation can be applied to fuzz test methods and any parameter type or subtype. By | ||
| * default, this annotation is propagated to all nested subtypes unless specified otherwise via the | ||
| * {@link #constraint()} attribute. | ||
| * | ||
| * <p>Example usage: | ||
| * | ||
| * <pre>{@code | ||
| * public class MyFuzzTarget { | ||
| * | ||
| * static Stream<?> dictionaryVisibleByAllArgumentMutators() { | ||
| * return Stream.of("example1", "example2", "example3", 1232187321, -182371); | ||
| * } | ||
| * | ||
| * static Stream<?> dictionaryVisibleOnlyByAnotherInput() { | ||
| * return Stream.of("code-intelligence.com", "secret.url.1082h3u21ibsdsazuvbsa.com"); | ||
| * } | ||
| * | ||
| * @DictionaryProvider("dictionaryVisibleByAllArgumentMutators") | ||
| * @FuzzTest | ||
| * public void fuzzerTestOneInput(String input, @DictionaryProvider("dictionaryVisibleOnlyByAnotherInput") String anotherInput) { | ||
| * // Fuzzing logic here | ||
| * } | ||
| * } | ||
| * }</pre> | ||
| * | ||
| * In this example, the mutator for the String parameter {@code input} of the fuzz test method | ||
| * {@code fuzzerTestOneInput} will be using the values returned by {@code provide} method during | ||
| * mutation, while the mutator for String {@code anotherInput} will use values from both methods: | ||
| * from the method-level {@code DictionaryProvider} annotation that uses {@code provide} and the | ||
| * parameter-level {@code DictionaryProvider} annotation that uses {@code provideSomethingElse}. | ||
| */ | ||
| @Target({ElementType.METHOD, TYPE_USE}) | ||
| @Retention(RUNTIME) | ||
| @IgnoreRecursiveConflicts | ||
| @PropertyConstraint | ||
| public @interface DictionaryProvider { | ||
| /** | ||
| * Specifies supplier methods that generate dictionary values for fuzzing the annotated method or | ||
| * type. The specified supplier methods must be static and return a {@code Stream <?>} of values. | ||
| * The values don't need to match the type of the annotated method or parameter exactly. The | ||
| * mutation framework will extract only the values that are compatible with the target type. | ||
| */ | ||
| String[] value() default {""}; | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The error message could be better if we remove the default value which forces users to specify |
||
|
|
||
| /** | ||
| * This {@code DictionaryProvider} will be used with probability {@code 1/p} by the mutator | ||
| * responsible for fitting types. Not all mutators respect this probability. | ||
| */ | ||
| int pInv() default 10; | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Would this make sense as a float setting? |
||
|
|
||
| /** | ||
| * Defines the scope of the annotation. Possible values are defined in {@link | ||
| * com.code_intelligence.jazzer.mutation.utils.PropertyConstraint}. It is convenient to use {@code | ||
| * RECURSIVE} as the default value here, as dictionary objects are typically used for complex | ||
| * types (e.g. custom classes) where the annotation is placed directly on the method or parameter | ||
| * and is expected to apply to all nested fields. | ||
| */ | ||
| String constraint() default RECURSIVE; | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,108 @@ | ||
| /* | ||
| * Copyright 2025 Code Intelligence GmbH | ||
| * | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| package com.code_intelligence.jazzer.mutation.support; | ||
|
|
||
| import static com.code_intelligence.jazzer.mutation.support.Preconditions.require; | ||
|
|
||
| import com.code_intelligence.jazzer.mutation.annotation.DictionaryProvider; | ||
| import com.code_intelligence.jazzer.mutation.runtime.MutationRuntime; | ||
| import java.lang.reflect.AnnotatedType; | ||
| import java.lang.reflect.InvocationTargetException; | ||
| import java.lang.reflect.Method; | ||
| import java.util.Arrays; | ||
| import java.util.HashSet; | ||
| import java.util.Map; | ||
| import java.util.Optional; | ||
| import java.util.stream.Collectors; | ||
| import java.util.stream.Stream; | ||
|
|
||
| public class DictionaryProviderSupport { | ||
|
|
||
| /** | ||
| * Extract inverse probability of the very first {@code DictionaryProvider} annotation on the | ||
| * given type. The {@code @DictionaryProvider} annotation directly on the type is preferred; if | ||
| * there is none, the first one appended because of {@code PropertyConstraint.RECURSIVE} is used. | ||
| * Any further {@code @DictionaryProvider} annotations appended later to this type because of | ||
| * {@code PropertyConstraint.RECURSIVE}, are ignored. Callers should ensure that at least one | ||
| * {@code @DictionaryProvider} annotation is present on the type. | ||
| */ | ||
| public static int extractFirstInvProbability(AnnotatedType type) { | ||
| // If there are several @DictionaryProvider annotations on the type, this will take the most | ||
| // immediate one, because @DictionaryProvider is not repeatable. | ||
| DictionaryProvider[] dictObj = type.getAnnotationsByType(DictionaryProvider.class); | ||
| if (dictObj.length == 0) { | ||
| // If we are here, it's a bug in the caller. | ||
| throw new IllegalStateException("Expected to find @DictionaryProvider, but found none."); | ||
| } | ||
| int pInv = dictObj[0].pInv(); | ||
| require(pInv >= 2, "@DictionaryProvider.pInv must be at least 2"); | ||
| return pInv; | ||
| } | ||
|
|
||
| /** Extract the provider streams using MutatorRuntime */ | ||
| public static Optional<Stream<?>> extractRawValues(AnnotatedType type) { | ||
| DictionaryProvider[] providers = | ||
| Arrays.stream(type.getAnnotations()) | ||
| .filter(a -> a instanceof DictionaryProvider) | ||
| .map(a -> (DictionaryProvider) a) | ||
| .toArray(DictionaryProvider[]::new); | ||
|
Comment on lines
+58
to
+62
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Wouldn't the |
||
| if (providers.length == 0) { | ||
| return Optional.empty(); | ||
| } | ||
| HashSet<String> providerMethodNames = | ||
| Arrays.stream(providers) | ||
| .map(DictionaryProvider::value) | ||
| .flatMap(Arrays::stream) | ||
| .filter(name -> !name.isEmpty()) | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Minor: Why filter out empty strings? IMO an error message that no such method is found would be cleaner. |
||
| .collect(Collectors.toCollection(HashSet::new)); | ||
| if (providerMethodNames.isEmpty()) { | ||
| return Optional.empty(); | ||
| } | ||
| Map<String, Method> fuzzTestMethods = | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The name of this variable threw me off when reading this. Could something like |
||
| Arrays.stream(MutationRuntime.fuzzTestMethod.getDeclaringClass().getDeclaredMethods()) | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Accessing global state in an unsuspecting |
||
| .filter(m -> m.getParameterCount() == 0) | ||
| .filter(m -> m.getReturnType().equals(Stream.class)) | ||
| .filter( | ||
| m -> | ||
| (m.getModifiers() & java.lang.reflect.Modifier.STATIC) | ||
| == java.lang.reflect.Modifier.STATIC) | ||
| .collect(Collectors.toMap(Method::getName, m -> m)); | ||
| return Optional.ofNullable( | ||
| providerMethodNames.stream() | ||
| .flatMap( | ||
| name -> { | ||
| Method m = fuzzTestMethods.get(name); | ||
| if (m == null) { | ||
| throw new IllegalStateException( | ||
| "Found no static supplier method 'Stream<?> " | ||
| + name | ||
| + "()' in class " | ||
| + MutationRuntime.fuzzTestMethod.getDeclaringClass().getName()); | ||
| } | ||
| try { | ||
| m.setAccessible(true); | ||
| return (Stream<?>) m.invoke(null); | ||
| } catch (IllegalAccessException e) { | ||
| throw new IllegalStateException("Cannot access method " + name, e); | ||
| } catch (InvocationTargetException e) { | ||
| throw new RuntimeException( | ||
| "Supplier method " + name + " threw an exception", e.getCause()); | ||
| } | ||
| }) | ||
| .distinct()); | ||
| } | ||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suspect that users might be confused on when to use the
DictionaryProviderand when to use other dictionary annotations likeDictionaryEntry,DictionaryFile."user-selected mutator types" might also not be descriptive enough and end with users annotating a
myFuzzTest(byte[] data)fuzz test with@DictionaryProviderwithout realizing that the values are not used.Would it make sense to add the
@DictionaryProvidervalues in addition to the libFuzzer dictionary file? Otherwise I could foresee situations where one wants to specify the same dictionary entries with@DictionaryProviderand@DictionaryEntry.