[RaftPeople]

Every step updates shared databases (frequently plural). In the case of the fulfillment step, the following systems+databases were involved: ERP, WMS, Shipping.

Typically, in end to end testing, tests are run within the same shared QA system and are semi-isolated based on choice of specific data (e.g. customers, products, orders, vendors, etc.). If this test causes a different test to fail, or vice-versa, then you have found a bug.

If we call that entire sequence of steps a "unit" test, would you start with testing the entire sequence of steps, or would you recommend testing the individual steps first?

And if we did test the individual steps first, we would give that testing a different name? Like maybe "sub-unit" testing?

[randomdata]

> Every step updates shared databases (frequently plural).

That's fine. It all happens within a single unit. A unit should mutate shared state within the unit. Testing would be pretty much useless without.

> If we call that entire sequence of steps a "unit" test, would you start with testing the entire sequence of steps, or would you recommend testing the individual steps first?

For all intents and purposes, you can't test the individual steps. All subsequent steps are dependent on the change in inventory state in step 1. And the product of step one is undoubtedly internal state, so there is no way for the test to observe the state change in isolation (unless you do something stupid). You have to carry out the subsequent steps to be able to infer that the inventory was, in fact, updated appropriately.

After all, the whole reason you are testing those steps together is because you recognize that they represent a single instance of functionality. You don't really get to choose (unless you choose to do something stupid, I suppose).

> And if we did test the individual steps first, we would give that testing a different name?

If the individual steps can be tested individually (ignoring a case of you doing something stupid), it's not actually and end-to-end process, so your example would make no sense. Granted, we have already questioned if it is a bad example.

[RaftPeople]

> For all intents and purposes, you can't test the individual steps.

Sure you can, and we did (that is a real example of an end to end test from a recent project) which also included testing the individual steps in isolation, which was preceded by testing the individual sub-steps/components of each step (which is the portion that is typically considered unit testing).

For example, step 1 is broken down into the following sub-steps which are all tested in isolation before testing the combined group together:

1.1-Calculate the current on hand inventory from all locations for all products

1.2-Calculate the current in transit inventory for all locations for all products

1.3-Calculate the current open inventory reservations by business partner and products

1.4-Calculate the current in process fulfillments by business partner and product

1.5-Resolve the configurable inventory feed rules for each business partner and product (or product group)

1.6-Using the data in 1.1 through 1.5, resolve the final available qty for each business partner and product

1.7-Construct system specific messages for each system and/or business partner (in some cases it's a one to one between business partner and system, but in other cases one system manages many business partners).

1.7.1-Send to system B

1.7.2-Send to system C

1.7.3-Send to system D

1.7.N-etc.

> And the product of step one is undoubtedly internal state, so there is no way for the test to observe the state change in isolation

The result of step 1 is that over in software system B (an entirely different application from system A) the inventory availability for each product from system A is properly represented in the system. Meaning queries, inquiries, reports, application functions (e.g. Inventory Availability by Partner), etc. all present the proper quantities.

To validate this step, it can be handled one of two ways:

1-Some sort of automated query that extracts data from system B and compares to the intended state from step 1 (probably by saving that data at the end of that step).

or 2-A user manually logs in to system B and compares to the expected values from step 1 (again saved or exposed in some way). This method works when the number of products is purposefully kept to a small number for testing purposes.

> If the individual steps can be tested individually (ignoring a case of you doing something stupid), it's not actually and end-to-end process, so your example would make no sense. Granted, we have already questioned if it is a bad example.

Yes the individual test can be tested in individually. Yes it is an end to end test.

> Granted, we have already questioned if it is a bad example.

It's a real example from a real project and it aligns with the general notion of an end to end test used in the industry.

More importantly, combined with the unit tests, functional tests, integration tests, performance tests, other end to end tests and finally user acceptance tests, it contributed to a successful go-live with very few bugs or design issues.