Test Harnesses
I was giving a talk a while back about the importance of a test harness when executing the strangulation pattern. It apparently was an unfamiliar topic to the audience: they had heard before about the techniques for strangling a domain, but had not spent any time discussing how to evaluate the correctness of the new, modern implementation.
Legacy implementations often do not have any scripted tests, let alone any automated unit tests to describe their behaviour. They work in production because they work in production: no one has touched them for a long time and no one has complained about them being wrong. If we are to create a new modern implementation, how can we be sure that we at least have parity with the legacy implementation?
Let’s start with a sample legacy implementation of a fibonacci calculation that lives in a Spring controller:
@Controller
public class FibController {
@GetMapping(“/fib”)
@ResponseBody
public int fib (@RequestParam(“number”) Integer number) throws FibException{
return fibCalc(number.intValue());
}
private int fibCalc (int number) throws FibException{
if (number < 0) {
throw new FibException (“invalid param”);
}
if (number == 0) {
return 0;
}
if (number == 1) {
return 1;
}
int twoBack = 0;
int lastSum = 1;
int newSum = 1;
for (int i = 2; i <= number; i++) {
newSum = twoBack + lastSum;
twoBack = lastSum;
lastSum = newSum;
}
return newSum;
}
}
Not the prettiest of code, but it gets the job done. I always try to assume best intentions when looking at old legacy implementations, as I’m sure to be embarrassed ten years from now by the code I write today.
Let’s build this in a more modern fashion, separating the controller from a service that does the work. Let’s start with the controller:
@Controller
@Slf4j
public class FibController {
FibService fibService;
public FibController(FibService fibService) {
this.fibService = fibService;
}
@GetMapping(“/fib”)
@ResponseBody
public Integer getByIndex(@RequestParam(“index”)Integer index) throws FibException {
log.info(“retrieving for index = “ + index);
return fibService.getAndSend(index.intValue(), uuid);
}
}
and also our service:
@Service
public class FibService {
public int get(int index) throws FibException {
if (index < 0) {
throw new FibException(“negative index supplied”);
}
if (index == 0) {
return 0;
}
if (index == 1) {
return 1;
}
return (get(index — 1) + get(index — 2));
}
}
We have a nice separation of concerns, and its very easy to write a unit test against our service implementation without having to establish an entire Spring context.
So if we want to strangle out the legacy implementation, the simplest thing to do would be to invoke the modern implementation from the legacy code base. And that would be fine. But we have no idea that our modern implementation is correct. Or at least that it returns the same values as our legacy implementation. And in a production system, changing the behaviour without any confidence in correctness can have some pretty catastrophic consequences.
So how do we get this confidence? A common pattern is to send some of the traffic to the new implementation alongside the legacy implementation. We can compare the new results against the old results to gain that confidence while still treating the legacy implementation as a source of truth.
So how much traffic should we send to the new version? And how can we do this in a way that allows us to quickly shut off that pipe if things go completely sideways?
We use a runtime configuration to manage this: the idea is that we can change the values of this configuration without having to restart (or redeploy!) the legacy application.
For this I used a simple h2 database table with a single row. Let’s start with the entity bean:
public class ControlsEntity {
@Id
private long id;
// if this is set, we use the result of the modern implementation if the request is forwarded there.
private boolean useModernImpl;
//what percent of requests to send to the modern implementation
private int sendToModernPercent;
}
There are two values we care about: what percent to send to the modern implementation, and whether or not to use the modern implementation as the source of truth. So early in our process we can send ten percent to the new impl, and still keep 100% of the results from the legacy system. If we flip that boolean, then we have 10% going to the modern version, and 90% using the legacy version as truth.
I then added a simple controller to allow POSTs to change this configuration at runtime:
@RestController
public class ControlsController {
private ControlsRespository controlsRespository;
public ControlsController(ControlsRespository controlsRespository) {
this.controlsRespository = controlsRespository;
}
@EventListener
public void appReady (ApplicationReadyEvent event) {
// initialize the config to send everything to the legacy implementation
controlsRespository.save(new ControlsEntity().withId(1L).withSendToModernPercent(0).withUseModernImpl(false));
}
@PostMapping(path = “/controls”,
consumes = MediaType.APPLICATION_JSON_VALUE,
produces = MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<ControlsEntity> changeControls (@RequestBody ControlsEntity controlsEntity) {
// we want exactly one control, and will reset the ID to ensure this.
controlsEntity.setId(1L);
controlsRespository.save(controlsEntity);
return new ResponseEntity<>(controlsEntity, HttpStatus.OK);
}
@GetMapping(“/controls”)
public ControlsEntity getControls () {
return controlsRespository.findById(1L).get();
}
}
The Repository in this instance is a simple CrudRepository.
I also added an Event Listener to seed the database on application start to send nothing to the modern implementation.
I should also point out that this example is completely unsecured: don’t do this in prod. Always secure endpoints with administrative permissions when exposing application configuration.
Now that we have runtime controls, let’s hook them into our Controller:
public FibController(ControlsRespository controlsRespository, ModernFibService modernFibService) {
this.controlsRespository = controlsRespository;
this.modernFibService = modernFibService;
}
private ControlsRespository controlsRespository;
private ModernFibService modernFibService;
private long lastLookupTime = 0;
ControlsEntity controlsEntity;
private static final long LOOKUP_INTERVAL = 10000;
@GetMapping(“/fib”)
@ResponseBody
public int fib (@RequestParam(“number”) Integer number) throws FibException{
long now = Calendar.getInstance().getTimeInMillis();
if (now — lastLookupTime > LOOKUP_INTERVAL) {
log.info(“looking up controls config”);
lastLookupTime = now;
controlsEntity = (ControlsEntity) controlsRespository.findById(1L).get();
}
log.info(“Controls Entity Value = “ + controlsEntity.toString());
double rando = Math.random()*100.0;
if ((rando < controlsEntity.getSendToModernPercent()) && controlsEntity.isUseModernImpl()){
//send to modern inline and return the value calculated
return modernFibService.sendToModern(number.intValue());
}
else if ((rando < controlsEntity.getSendToModernPercent()) && (!controlsEntity.isUseModernImpl())) {
// send to modern async
log.debug (“sending async, controller thread is “ + Thread.currentThread().getName());
modernFibService.sendToModernAsync(number.intValue());
}
return fibCalc(number.intValue());
}
let’s examine this code more closely. At the start of the method, we do a time check against a LOOKUP_INTERVAL. if we’re past that interval, we lookup the controls configuration, otherwise we just use whatever the last lookup value was. This acts as a simple cache to prevent disk lookups on every invocation.
Next we randomize our “send to modern” decision, as opposed to sequencing it. Getting the percentage to send exactly correct isn’t important: the idea is to get close enough to trust our new implementation.
Lastly we send to the modern implementation in one of two ways: either asynchronously or synchronously. If we care about the result, we should wait until that result has been returned from the modern version and return that to our client. But if we are just invoking it and don’t care about returning that result to our client, there’s no need to block on that invocation. We can do it in another thread. Let’s take a look at the ModernFibService code:
@Service
@Slf4j
public class ModernFibService {
RestTemplate restTemplate = new RestTemplate();
@Async
public void sendToModernAsync (int number) {
log.debug(“In async method, thread name is “ + Thread.currentThread().getName());
sendToModern(number);
}
public int sendToModern (int number) {
log.info(“sending to the modern implementation for a result”);
String uuid = UUID.randomUUID().toString();
ResponseEntity<Integer> response =
restTemplate.getForEntity(“http://localhost:8082/fib?index=” + number + “&uuid=” + uuid, Integer.class);
return response.getBody().intValue();
}
}
I just hard-coded the modern url here for simplicity’s sake.
Readers may note that we added a parameter to the invocation: a UUID. This allows us to correlate a request from the legacy system with a request to the modern system. This becomes important later as we start doing comparisons against large sets of data.
So at this point, we have a legacy system that will conditionally invoke a modern system that _should_ be returning the same results, and we have the ability to change those conditions at runtime. The next step is to send our results (both legacy and modern) somewhere where we can do some checking.
Messaging is a great solution for this. Putting a message on an exchange is a relatively lightweight method of dumping interesting data. We can then use a collector application to gather all the results and put those somewhere where we can do analysis. Let’s start with the code in the legacy implementation. We just change our ModernFibService to write a message:
@Service
@Slf4j
public class ModernFibService {
private StreamBridge streamBridge;
RestTemplate restTemplate = new RestTemplate();
public ModernFibService(StreamBridge streamBridge) {
this.streamBridge = streamBridge;
}
@Async
public void sendToModernAsync (int number) {
log.debug(“In async method, thread name is “ + Thread.currentThread().getName());
sendToModern(number);
}
public int sendToModern (int number) {
log.info(“sending to the modern implementation for a result”);
String uuid = UUID.randomUUID().toString();
ResponseEntity<Integer> response =
restTemplate.getForEntity(“http://localhost:8082/fib?index=” + number + “&uuid=” + uuid, Integer.class);
int newVal = response.getBody().intValue();
streamBridge.send(“fibcalc”, new FibCalc().withIndex(number).withUuid(uuid).withValue(newVal).withSource(“LEGACY”));
return newVal;
}
}
I’m using Spring Cloud Streams and the StreamBridge implementation here. A full discussion of that is out of scope for this post, but I encourage you to read more about it if this is a new topic.
From the modern side, we had to do a little refactoring. We renamed the FibService to a FibCalcService, and added the messaging to a FibService:
@Service
public class FibService {
private final StreamBridge streamBridge;
private final FibCalcService fibCalcService;
public FibService(StreamBridge streamBridge, FibCalcService fibCalcService) {
this.streamBridge = streamBridge;
this.fibCalcService = fibCalcService;
}
public int getAndSend(int index, String uuid) throws FibException {
int newVal = fibCalcService.get(index);
sendResult(new FibCalc().withIndex(index).withValue(newVal).withUuid(uuid).withSource(“MODERN”));
return newVal;
}
private void sendResult(FibCalc fibCalc) {
streamBridge.send(“fibcalc”, fibCalc);
}
}
and changed the FibController to include a UUID:
@GetMapping(“/fib”)
@ResponseBody
public Integer getByIndex(@RequestParam(“index”)Integer index, @RequestParam(“uuid”)String uuid) throws FibException {
log.info(“retrieving for index = “ + index);
return fibService.getAndSend(index.intValue(), uuid);
}
You’ll note that the same UUID is used in both calculations, and we’ve tagged the message in both implementations to include the index requested, value calculated, UUID, and source. This bookkeeping should make it easy to do some analysis.
The last thing we have to do is build our test harness application: this will collect the messages, write them to a repository, and expose some endpoints for queries that tell us the correctness of our new implementation. Let’s start with the listener:
@Service
@Slf4j
public class FibCalcResultReceiver {
private ResultRepository resultRepository;
public FibCalcResultReceiver(ResultRepository resultRepository) {
this.resultRepository = resultRepository;
}
@Bean
public Consumer<FibCalc> resultRecieved () {
return fibCalc -> {
log.info(“Received: {}”, fibCalc.toString());
resultRepository.save(new FibCalcEntity()
.withCalculatedValue(fibCalc.getValue())
.withIndex(fibCalc.getIndex())
.withSource(fibCalc.getSource())
.withUuid(fibCalc.getUuid())
);
};
}
}
All we do here is log the message and write it to our repo. That repo has a couple custom queries:
public interface ResultRepository extends JpaRepository<FibCalcEntity, Long> {
@Query(“select count(distinct f.uuid) from FibCalcEntity f”)
long countDistinctUuids();
@Query(“select count(f1.uuid) from FibCalcEntity f1, FibCalcEntity f2” +
“ where f1.uuid = f2.uuid” +
“ and f1.source=’LEGACY’ and f2.source=’MODERN’” +
“ and f1.calculatedValue = f2.calculatedValue”)
long countGoodMatches();
@Query(“select f2 as modernResult, f1 as legacyResult from FibCalcEntity f1, FibCalcEntity f2” +
“ where f1.uuid = f2.uuid” +
“ and f1.source=’LEGACY’ and f2.source=’MODERN’” +
“ and not f1.calculatedValue = f2.calculatedValue”)
List<FibCalcEntityPair> getBadMatches();
}
The first tells us how many request pairs (the pair being modern and legacy implementations) this service has collected. The second tells us how many of those pairs have identical calculated values. The third returns the pairs where the calculated values are different. Lastly, we expose these queries through a controller:
@RestController
@Slf4j
public class FibCalcController {
ResultRepository resultRepository;
public FibCalcController(ResultRepository resultRepository) {
this.resultRepository = resultRepository;
}
@GetMapping(“/all”)
public List<FibCalcEntity> getAll() {
return resultRepository.findAll();
}
@GetMapping(“/totalCalcCount”)
public long getTotalCalcCount() {
return resultRepository.countDistinctUuids();
}
@GetMapping(“/goodCountRatio”)
public double getTotalGoodCalcCountPercent() {
long totalCount = resultRepository.countDistinctUuids();
long goodCount = resultRepository.countGoodMatches();
log.info(“good count, total count = “ + goodCount + “, “ + totalCount);
return (double)goodCount/(double)totalCount;
}
@GetMapping(“/badMatches”)
public List<FibCalcEntityPair> getBadMatches() {
return resultRepository.getBadMatches();
}
}
Again, this is an example and not meant for production. Please, please, please secure your endpoints.
Our Final system looks like this:
So now we’re in pretty good shape! We have a legacy implementation that can conditionally invoke a modern application. When it does, it writes the results of its calculations to an exchange. The modern implementation also writes its results to that same exchange, and we’ve built a test harness that collects all these results and exposes some basic queries to help us demonstrate the correctness of our results. That demonstration can go a long way in getting your customer to be confident that your new, strangled domain has parity with the old implementation. And once you have that parity, you can begin to re-imagine how the system should behave.
One last note: Even though this is an example, I didn’t spend a lot of time on reliability patterns here. And that was on purpose. Investing in reliability is an investment away from feature development. Does reliability matter very much here for our test harness? Is it the end of the world if we drop one of these messages? Probably not. Is this code going to live forever? Absolutely not: we can delete it as soon as we’re confident in our new implementation. Investments in throw-away code may not be the best use of your development time.
Code for this exercise can be found here, with a README to help you run this locally:
