|
|
|
|
|
|
by slunk
1776 days ago
|
|
|
> It has nothing to do with what the model is being used for. I may be misunderstanding this passage of the article, but I thought the author was claiming that machine learning (specifically training) was equivalent to compression, while language understanding is equivalent to decompression. Therefore, they can't be the same thing. Why does language understand have to be analogous to training an ML model rather than using an ML model for inference? |
|
|
Why would you look at ML model inferences in particular? There is no compression or decompression going on during inferences, you're just running data through the existing weights.
Creating an ML model on the other hand is lossy compression. You reduce the size of the data (Training set -> model) in exchange for reduced accuracy (100% -> 90-95% or whatever).
NLU is decompression because you are extracting information that doesn't exist in the text.
I see ML as ahead-of-time compression (Creating a model), whereas NLU is just-in-time decompression (Extracting information from current context). Looking specifically at inference-time doesn't make sense to me because all the work for ML is done during training, not inference.