Hacker News new | ask | show | jobs
by jacobr1 2757 days ago
I use `refresh=True` on my insert/update/delete, which forces the writes to complete. Then all the reads work as you would expect.
1 comments
kn7 2757 days ago
That still doesn't guarantee that a consecutive read is gonna get the last state.
link
haggy 2757 days ago
Direct from ES documentation on the `refresh` flag: "Refresh the relevant primary and replica shards (not the whole index) immediately after the operation occurs, so that the updated document appears in search results immediately."

Do you have other information about the refresh flag because their documentation clearly states that forcing a refresh is applied to primary and replica shards meaning that it will be available for query directly after the call to refresh is made.

link
JohnBooty 2754 days ago
My experience was that this was not reliable. It was nearly always true, but not always, and tests would sporadically fail some small percentage of the time.

However, this was back in ~2015 and Elasticsearch 1.3 or something like this, which is of course a now-ancient version. Perhaps things are different now.

edit: Perhaps we were using the refresh command and not the refresh flag. It was a few years ago and I don't have access to the code any more, and my memory may be failing here. If the refresh flag works as advertised (enforces an index update and guarantees a consistent view of the data for the next query, which the command did not seem to) then that of course solves my initial problem W.R.T. writing tests.

link
bobbyi_settv 2757 days ago
Yes it does. I'm not sure what you think the refresh operation is if not that.

I've run countless integration tests with ES and never seen something fail due to refresh not working as advertised. If you have, what version of ES was it? Can you give some sample code that sporadically exhibits the problem?

link
kn7 2757 days ago
We generally use index refresh in ITs (running ES in Docker) and it fails occasionally, which I believe the case described here: "The (near) real-time capabilities depend on the index engine used." https://www.elastic.co/guide/en/elasticsearch/reference/curr...
link
manigandham 2757 days ago
That seems to be the issue then. The refresh flag should be passed in your insert/update/delete operations.

The refresh command can also be called (which is what you're doing) but this is a different operation and just triggers the index build with no guarantees that it finishes or is consistent with any particular data mutation.

Did you read the previously posted documentation for the refresh flag?

link