[agolio]

> I then push them on this until they realize that any implementation of the dictionary lookup function of a string would be o(n) at best, and as such the total runtime of the solution is O(n²).

Is that true?

I am actually siding towards the candidates on this one

presumably it would be O(n_words_in_dictionary) for the dictionary check, which as the number of words in the dictionary is constant, would make the overall algorithm O(n) ...

[behdad]

You still need to query every byte of the input string...

Any better than that you can do would be O(min(n, k)) where k is the length of the longest word in the dictionary. But without loss of generality that would be O(n).

[abtinf]

You only need to do that if the length of words is unbound, which would be nonsensical.

The maximum number of iterations is length of the largest word in the dictionary.

[agolio]

Good point, though I guess the point of the disagreement is then for me that the question's "dictionary words" implies to me an english dictionary, rather than an abstract dictionary of words of potentially infinite length

[d1sxeyes]

It would be impossible to look up anything in a dictionary of infinite length.

Therefore, either the question is nonsensical, or there must exist a word (or words) of a maximum length.

[activitypea]

You either care about the order of complexity, which is the big O, or you care about real-world performance of byte level operations. You can't have it both ways.

[light_hue_1]

I don't think so.

You take the first n bytes of your input string. You hash them. You look them up in the dictionary. The lookup is O(1) but hashing your input is O(n).

But there's a simple way around this.

We have incremental hash functions. When you add a byte to the string and you use an incremental hash, you do a fixed amount of extra work to update the hash. Like that you only need to O(1) extra work to compute the new hash.

One trivial incremental hash is to break your input into fixed size chunks, hash each chunk, and xor the chunks together.

[behdad]

That's the same as the trie solution.

[light_hue_1]

It's not at all like the trie solution! Not in performance and certainly not in terms of complexity.

A trie is a pretty nasty datastructure memory wise. It involves chasing pointers all over the place. The constants on a trie are not going to be very good.

For an incremental hash, you hash your string, and then you fix your hash up as you go. Then you look up your hash in a hash table. Code-wise it's far simpler to implement. And performance wise it will be far faster!

The two solutions have nothing to do with one another. Except that they're both O(n).

[evouga]

What you say is true, but the rolling hash solution is susceptible to potential hash collisions while the trie is not.

[light_hue_1]

Ok sure. So what?

Hash collisions don't matter unless they're pathological. Just choose an out of the box hash that works well.

The article is just wrong.