Hacker News new | ask | show | jobs
by pitah1 889 days ago
Interesting feature. One key thing I found when testing is that for you to reproduce the set of steps the user went through, there are a data attribute(s) that need to remain the same. For example, after login, a request for your account information will contain an account_id number that should be the same for all other account requests. If you can't guarantee this, then I don't see how you could use this in any sort of integration tests.

Isn't it simpler to use the Open API spec then generate from there?

1 comments

Hi @pitah1, thanks for the comment! I'm Jon, the guy who built the feature :)

You're spot on here. Unique identifiers within a flow shouldn't necessarily be replaced with a randomly generated string. We've attacked this problem from two different angles.

1. If an API request has a JSON response, we'll generate JS load test code for that request that begins with `const responseA = await axios[...];`. You can edit the load test to use the response data in subsequent requests.

2. We also attempt to intelligently replace UUIDs (or predictable identifiers) with a placeholder like `{fieldName}`. This highlights the values that need user intervention.

We use the Swagger schema to determine the available API endpoints, if each endpoint has a request body vs query string params vs none, and so on. We sprinkle in AI to help decide how to best saturate request bodies with realistic data via faker.js.