Skip to content

Commit af4474a

Browse files
dirkboltedirkbolte
authored
bug!: fix updateCount handling (#110)
- introduce a pseudo-transaction which writes updates when a request ends - introduce a transactionManager for this - move locking to transactionManager (and remove some unneeded locks) - update tests as the updateCount is only increased once now <!-- Please describe your pull request here. --> ## References #102 <!-- References to relevant GitHub issues and pull requests, esp. upstream and downstream changes --> ## Submitter checklist - [ ] Recommended: Join [WireMock Slack](https://slack.wiremock.org/) to get any help in `#help-contributing` or a project-specific channel like `#wiremock-java` - [ ] The PR request is well described and justified, including the body and the references - [ ] The PR title represents the desired changelog entry - [ ] The repository's code style is followed (see the contributing guide) - [ ] Test coverage that demonstrates that the change works as expected - [ ] For new features, there's necessary documentation in this pull request or in a subsequent PR to [wiremock.org](https://github.com/wiremock/wiremock.org) <!-- Put an `x` into the [ ] to show you have filled the information. The template comes from https://github.com/wiremock/.github/blob/main/.github/pull_request_template.md You can override it by creating .github/pull_request_template.md in your own repository -->
1 parent 14b062f commit af4474a

23 files changed

+676
-327
lines changed

‎README.md‎

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -311,7 +311,7 @@ The state is recorded in `serveEventListeners` of a stub. The following function
311311
- to delete a selective property, set it to `null` (as string).
312312
- `list` : stores a state in a list. Can be used to prepend/append new states to an existing list. List elements cannot be modified (only read/deleted).
313313

314-
`state` and `list` can be used in the same `ServeEventListener` (would count as two updates). Adding multiple `recordState` `ServeEventListener` is supported.
314+
`state` and `list` can be used in the same `ServeEventListener` (would count as ONE updates). Adding multiple `recordState` `ServeEventListener` is supported.
315315

316316
The following parameters have to be provided:
317317

@@ -803,7 +803,9 @@ For documentation on using these matchers, check the [WireMock documentation](ht
803803

804804
### Context update count match
805805

806-
Whenever the serve event listener `recordState` is processed, the internal context update counter is increased. The number can be used
806+
Whenever a request with a serve event listener `recordState` or `deleteState` is processed, the internal context update counter is increased.
807+
The update count is increased by one whenever there is at least one change to a context (so: property adding/change, list entry addition/deletion). Multiple
808+
event listeners with multiple changes of a single context within a single request only result in an increase by one.
807809
for request matching as well. The following matchers are available:
808810

809811
- `updateCountEqualTo`
@@ -1070,6 +1072,20 @@ body file with handlebars to ignore a missing property:
10701072
}
10711073
```
10721074

1075+
# Distributed setups and concurrency
1076+
1077+
This extension is at the moment not optimized for distributed setups or high degrees concurrency. While it will basically work, there are some limitations
1078+
that should be held into account:
1079+
1080+
- The store used for storing the state is on instance-level only
1081+
- while it can be exchanged for a distributed store, any atomicity assurance on instance level is not replicated to the distributed setup. Thus concurrent operations on different instances might result in state overwrites
1082+
- Lock-level is basically the whole context store
1083+
- while the lock time is kept small, this can still impact measurements when being used in load tests
1084+
- Single updates to contexts (property additions or changes, list entry additions or deletions) are atomic on instance level
1085+
- Concurrent requests are currently allowed to change the same context. Atomicity prevents overwrites but does not provide something like a transaction, so: the context can change while a request is performed
1086+
1087+
For any kind of usage with parallel write requests, it's recommended to use a different context for each parallel stream.
1088+
10731089
# Debugging
10741090

10751091
In general, you can increase verbosity, either by [register a notifier](https://wiremock.org/3.x/docs/configuration/#notification-logging)

‎src/main/java/org/wiremock/extensions/state/StateExtension.java‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,9 @@
2424
import org.wiremock.extensions.state.extensions.RecordStateEventListener;
2525
import org.wiremock.extensions.state.extensions.StateRequestMatcher;
2626
import org.wiremock.extensions.state.extensions.StateTemplateHelperProviderExtension;
27+
import org.wiremock.extensions.state.extensions.TransactionEventListener;
2728
import org.wiremock.extensions.state.internal.ContextManager;
29+
import org.wiremock.extensions.state.internal.TransactionManager;
2830

2931
import java.util.Collections;
3032
import java.util.List;
@@ -52,15 +54,18 @@ public class StateExtension implements ExtensionFactory {
5254
private final StateTemplateHelperProviderExtension stateTemplateHelperProviderExtension;
5355
private final RecordStateEventListener recordStateEventListener;
5456
private final DeleteStateEventListener deleteStateEventListener;
57+
private final TransactionEventListener transactionEventListener;
5558
private final StateRequestMatcher stateRequestMatcher;
5659

5760
public StateExtension(Store<String, Object> store) {
58-
var contextManager = new ContextManager(store);
61+
var transactionManager = new TransactionManager(store);
62+
var contextManager = new ContextManager(store, transactionManager);
5963
this.stateTemplateHelperProviderExtension = new StateTemplateHelperProviderExtension(contextManager);
6064
var templateEngine = new TemplateEngine(stateTemplateHelperProviderExtension.provideTemplateHelpers(), null, Collections.emptySet(), false);
6165

6266
this.recordStateEventListener = new RecordStateEventListener(contextManager, templateEngine);
6367
this.deleteStateEventListener = new DeleteStateEventListener(contextManager, templateEngine);
68+
this.transactionEventListener = new TransactionEventListener(transactionManager);
6469
this.stateRequestMatcher = new StateRequestMatcher(contextManager, templateEngine);
6570
}
6671

@@ -69,6 +74,7 @@ public List<Extension> create(WireMockServices services) {
6974
return List.of(
7075
recordStateEventListener,
7176
deleteStateEventListener,
77+
transactionEventListener,
7278
stateRequestMatcher,
7379
stateTemplateHelperProviderExtension
7480
);

‎src/main/java/org/wiremock/extensions/state/extensions/DeleteStateEventListener.java‎

Lines changed: 113 additions & 96 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,9 @@
2424
import com.github.tomakehurst.wiremock.stubbing.ServeEvent;
2525
import org.apache.commons.lang3.StringUtils;
2626
import org.wiremock.extensions.state.internal.ContextManager;
27-
import org.wiremock.extensions.state.internal.DeleteStateParameters;
28-
import org.wiremock.extensions.state.internal.ResponseTemplateModel;
2927
import org.wiremock.extensions.state.internal.StateExtensionMixin;
28+
import org.wiremock.extensions.state.internal.api.DeleteStateParameters;
29+
import org.wiremock.extensions.state.internal.model.ResponseTemplateModel;
3030

3131
import java.util.List;
3232
import java.util.Map;
@@ -72,122 +72,139 @@ public void beforeResponseSent(ServeEvent serveEvent, Parameters parameters) {
7272
"response", ResponseTemplateModel.from(serveEvent.getResponse())
7373
);
7474
var configuration = Json.mapToObject(parameters, DeleteStateParameters.class);
75-
Optional.ofNullable(configuration.getList()).ifPresentOrElse(
76-
listConfig -> handleListDeletion(listConfig, createContextName(model, configuration.getContext()), model),
77-
() -> handleContextDeletion(configuration, model)
78-
);
75+
new ListenerInstance(serveEvent.getId().toString(), model, configuration).run();
7976
}
8077

81-
private void handleContextDeletion(DeleteStateParameters configuration, Map<String, Object> model) {
82-
if (configuration.getContext() != null) {
83-
deleteContext(configuration.getContext(), model);
84-
} else if (configuration.getContexts() != null) {
85-
deleteContexts(configuration.getContexts(), model);
86-
} else if (configuration.getContextsMatching() != null) {
87-
deleteContextsMatching(configuration.getContextsMatching(), model);
88-
} else {
89-
throw createConfigurationError("Missing/invalid configuration for context deletion");
90-
}
78+
private String renderTemplate(Object context, String value) {
79+
return templateEngine.getUncachedTemplate(value).apply(context);
9180
}
9281

93-
private void deleteContexts(List<String> rawContexts, Map<String, Object> model) {
82+
private class ListenerInstance {
83+
private final String requestId;
84+
private final DeleteStateParameters configuration;
85+
private final Map<String, Object> model;
9486

95-
var contexts = rawContexts.stream().map(it -> renderTemplate(model, it)).collect(Collectors.toList());
96-
contextManager.onEach(context -> {
97-
if(contexts.contains(context.getContextName())) {
98-
contextManager.deleteContext(context.getContextName());
87+
ListenerInstance(String requestId, Map<String, Object> model, DeleteStateParameters configuration) {
88+
this.requestId = requestId;
89+
this.model = model;
90+
this.configuration = configuration;
91+
}
92+
93+
public void run() {
94+
Optional.ofNullable(configuration.getList()).ifPresentOrElse(
95+
listConfig -> handleListDeletion(listConfig, createContextName(configuration.getContext())),
96+
this::handleContextDeletion
97+
);
98+
}
99+
100+
private void handleContextDeletion() {
101+
if (configuration.getContext() != null) {
102+
deleteContext(configuration.getContext());
103+
} else if (configuration.getContexts() != null) {
104+
deleteContexts(configuration.getContexts());
105+
} else if (configuration.getContextsMatching() != null) {
106+
deleteContextsMatching(configuration.getContextsMatching());
107+
} else {
108+
throw createConfigurationError("Missing/invalid configuration for context deletion");
99109
}
100-
});
101-
}
110+
}
111+
112+
private void deleteContexts(List<String> rawContexts) {
102113

103-
private void deleteContextsMatching(String rawRegex, Map<String, Object> model) {
104-
try {
105-
var regex = renderTemplate(model, rawRegex);
106-
var pattern = Pattern.compile(regex);
107-
contextManager.onEach(context -> {
108-
if(pattern.matcher(context.getContextName()).matches()) {
109-
contextManager.deleteContext(context.getContextName());
114+
var contexts = rawContexts.stream().map(it -> renderTemplate(model, it)).collect(Collectors.toList());
115+
contextManager.onEach(requestId, context -> {
116+
if (contexts.contains(context.getContextName())) {
117+
contextManager.deleteContext(requestId, context.getContextName());
110118
}
111119
});
112-
} catch (PatternSyntaxException ex) {
113-
throw createConfigurationError("Missing/invalid configuration for context deletion: %s", ex.getMessage());
114120
}
115-
}
116121

117-
private void deleteContext(String rawContext, Map<String, Object> model) {
118-
contextManager.deleteContext(createContextName(model, rawContext));
119-
}
122+
private void deleteContextsMatching(String rawRegex) {
123+
try {
124+
var regex = renderTemplate(model, rawRegex);
125+
var pattern = Pattern.compile(regex);
126+
contextManager.onEach(requestId, context -> {
127+
if (pattern.matcher(context.getContextName()).matches()) {
128+
contextManager.deleteContext(requestId, context.getContextName());
129+
}
130+
});
131+
} catch (PatternSyntaxException ex) {
132+
throw createConfigurationError("Missing/invalid configuration for context deletion: %s", ex.getMessage());
133+
}
134+
}
120135

121-
private void handleListDeletion(DeleteStateParameters.ListParameters listConfig, String contextName, Map<String, Object> model) {
122-
if (Boolean.TRUE.equals(listConfig.getDeleteFirst())) {
123-
deleteFirst(contextName);
124-
} else if (Boolean.TRUE.equals(listConfig.getDeleteLast())) {
125-
deleteLast(contextName);
126-
} else if (StringUtils.isNotBlank(listConfig.getDeleteIndex())) {
127-
deleteIndex(listConfig, contextName, model);
128-
} else if (listConfig.getDeleteWhere() != null &&
129-
listConfig.getDeleteWhere().getProperty() != null &&
130-
listConfig.getDeleteWhere().getValue() != null
131-
) {
132-
deleteWhere(listConfig, contextName, model);
133-
} else {
134-
throw createConfigurationError("Missing/invalid configuration for list entry deletion");
136+
private void deleteContext(String rawContext) {
137+
contextManager.deleteContext(requestId, createContextName(rawContext));
135138
}
136-
}
137139

138-
private Long deleteFirst(String contextName) {
139-
return contextManager.createOrUpdateContextList(contextName, maps -> {
140-
if (!maps.isEmpty()) maps.removeFirst();
141-
logger().info(contextName, "list::deleteFirst");
142-
});
143-
}
140+
private void handleListDeletion(DeleteStateParameters.ListParameters listConfig, String contextName) {
141+
if (Boolean.TRUE.equals(listConfig.getDeleteFirst())) {
142+
deleteFirst(contextName);
143+
} else if (Boolean.TRUE.equals(listConfig.getDeleteLast())) {
144+
deleteLast(contextName);
145+
} else if (StringUtils.isNotBlank(listConfig.getDeleteIndex())) {
146+
deleteIndex(listConfig, contextName);
147+
} else if (listConfig.getDeleteWhere() != null &&
148+
listConfig.getDeleteWhere().getProperty() != null &&
149+
listConfig.getDeleteWhere().getValue() != null
150+
) {
151+
deleteWhere(listConfig, contextName);
152+
} else {
153+
throw createConfigurationError("Missing/invalid configuration for list entry deletion");
154+
}
155+
}
144156

145-
private void deleteLast(String contextName) {
146-
contextManager.createOrUpdateContextList(contextName, maps -> {
147-
if (!maps.isEmpty()) maps.removeLast();
148-
logger().info(contextName, "list::deleteLast");
149-
});
150-
}
157+
private void deleteFirst(String contextName) {
158+
contextManager.createOrUpdateContextList(requestId, contextName, maps -> {
159+
if (!maps.isEmpty()) maps.removeFirst();
160+
logger().info(contextName, "list::deleteFirst");
161+
});
162+
}
151163

152-
private void deleteIndex(DeleteStateParameters.ListParameters listConfig, String contextName, Map<String, Object> model) {
153-
try {
154-
var index = Integer.parseInt(renderTemplate(model, listConfig.getDeleteIndex()));
155-
contextManager.createOrUpdateContextList(contextName, list -> {
156-
list.remove(index);
157-
logger().info(contextName, String.format("list::deleteIndex(%d)", index));
164+
private void deleteLast(String contextName) {
165+
contextManager.createOrUpdateContextList(requestId, contextName, maps -> {
166+
if (!maps.isEmpty()) maps.removeLast();
167+
logger().info(contextName, "list::deleteLast");
158168
});
159-
} catch (IndexOutOfBoundsException | NumberFormatException e) {
160-
logger().info(contextName, String.format("Unknown or unparsable list index: '%s' - ignoring", listConfig.getDeleteIndex()));
161169
}
162-
}
163170

164-
private void deleteWhere(DeleteStateParameters.ListParameters listConfig, String contextName, Map<String, Object> model) {
165-
var property = renderTemplate(model, listConfig.getDeleteWhere().getProperty());
166-
var value = renderTemplate(model, listConfig.getDeleteWhere().getValue());
167-
contextManager.createOrUpdateContextList(contextName, list -> {
168-
var iterator = list.iterator();
169-
while (iterator.hasNext()) {
170-
var element = iterator.next();
171-
if (Objects.equals(element.getOrDefault(property, null), value)) {
172-
iterator.remove();
173-
logger().info(contextName, String.format("list::deleteWhere(property=%s)", property));
174-
break;
175-
}
171+
private void deleteIndex(DeleteStateParameters.ListParameters listConfig, String contextName) {
172+
try {
173+
var index = Integer.parseInt(renderTemplate(model, listConfig.getDeleteIndex()));
174+
contextManager.createOrUpdateContextList(requestId, contextName, list -> {
175+
list.remove(index);
176+
logger().info(contextName, String.format("list::deleteIndex(%d)", index));
177+
});
178+
} catch (IndexOutOfBoundsException | NumberFormatException e) {
179+
logger().info(contextName, String.format("Unknown or unparsable list index: '%s' - ignoring", listConfig.getDeleteIndex()));
176180
}
177-
});
178-
}
181+
}
179182

180-
private String createContextName(Map<String, Object> model, String rawContext) {
181-
var context = Optional.ofNullable(rawContext).filter(StringUtils::isNotBlank)
182-
.map(it -> renderTemplate(model, it))
183-
.orElseThrow(() -> new ConfigurationException("no context specified"));
184-
if (StringUtils.isBlank(context)) {
185-
throw createConfigurationError("Context cannot be blank");
183+
private void deleteWhere(DeleteStateParameters.ListParameters listConfig, String contextName) {
184+
var property = renderTemplate(model, listConfig.getDeleteWhere().getProperty());
185+
var value = renderTemplate(model, listConfig.getDeleteWhere().getValue());
186+
contextManager.createOrUpdateContextList(requestId, contextName, list -> {
187+
var iterator = list.iterator();
188+
while (iterator.hasNext()) {
189+
var element = iterator.next();
190+
if (Objects.equals(element.getOrDefault(property, null), value)) {
191+
iterator.remove();
192+
logger().info(contextName, String.format("list::deleteWhere(property=%s)", property));
193+
break;
194+
}
195+
}
196+
});
197+
}
198+
199+
private String createContextName(String rawContext) {
200+
var context = Optional.ofNullable(rawContext).filter(StringUtils::isNotBlank)
201+
.map(it -> renderTemplate(model, it))
202+
.orElseThrow(() -> new ConfigurationException("no context specified"));
203+
if (StringUtils.isBlank(context)) {
204+
throw createConfigurationError("Context cannot be blank");
205+
}
206+
return context;
186207
}
187-
return context;
188-
}
189208

190-
private String renderTemplate(Object context, String value) {
191-
return templateEngine.getUncachedTemplate(value).apply(context);
192209
}
193210
}

0 commit comments

Comments
 (0)