[stickperson]

Oh, I think I get it now-- is it that the _initial_ indexing is lazy, but all indexing after that is done automatically by the historical index workers? Basically when a user searches for something do you check that ES has something for that user, if it doesn't start off the initial indexing process, and from there the workers do their thing?

[jhgg]

The historical index workers index the history of the server, whereas we have a real-time index worker that is consuming and bulk inserting messages in real time. Searching for the first time in a server turns on real-time indexing and triggers a historical index of all previous messages in that server.

[sandGorgon]

Can you talk about the bulk insert for real time messages ? How does this get triggered - does it run every X minutes or X messages (your code looks like it is every X messages).

Are you using DB triggers to fire the job ?

[stickperson]

Got it. I was under the impression that all messages were lazily indexed. After reading the article again it's pretty clear that's not the case. Thanks for the clarification.