|
I understand that this discussion can become frustrating, especially when you see repetitive patterns in the discourse or feel like the nuances are not being sufficiently addressed. However, there are a few points I would like to clarify: Semantics in word embeddings: While I agree that word embeddings cannot fully capture human-like semantic understanding, they do provide a mathematical representation that has proven useful in many NLP tasks. It's not that word embeddings "understand" semantics in the human sense, but they do capture certain aspects of meaning that are statistically derived from their use in the training corpus. This is not an unsubstantiated claim. It is empirically demonstrated in numerous tasks where semantic understanding is beneficial, like semantic similarity, word analogy, and other downstream tasks such as translation, sentiment analysis, text classification, etc. Your point about BERT overfitting to statistical regularities of the dataset is well taken. Indeed, it exposes the limitations of the model and the need for careful design and evaluation of benchmarks. However, it's worth noting that a failure in one specific test doesn't invalidate the successes in other tasks. It simply highlights an area that needs improvement. It's true that there's a flood of papers and not all of them have substantial novelty or impact. This is not a problem exclusive to deep learning or AI, but a broader issue in academia and scientific publishing. However, amidst the noise, there's also a lot of valuable work being done, with genuine advancements and novel approaches. You mentioned that connectionists only improve their systems when someone rubs their face in it. This is essentially how scientific progress happens - through skepticism, criticism, and the relentless pursuit of truth. I would argue that the current era of deep learning research is no different. It's a messy, iterative process, with steps forward, backward, and sideways. Furthermore, I believe it's crucial to remember that there's room for both connectionist and symbolic approaches in AI. It's not necessarily a matter of one being 'right' and the other 'wrong.' Rather, they offer different perspectives and techniques that can be valuable in different contexts. Connectionist models, like the neural networks we've been discussing, are incredibly effective at tasks like pattern recognition and prediction, especially when dealing with large, high-dimensional datasets. On the other hand, symbolic models are excellent at representing explicit knowledge and reasoning logically, making them useful for tasks that require a high degree of interpretability or strict adherence to predefined rules. The future of AI likely involves finding ways to integrate these two approaches, leveraging the strengths of each to overcome their respective limitations. The field is vast and diverse, and there's plenty of room for different methods and viewpoints. PS: I understand where you're coming from. Sometimes I need a break from this too. Remember there is no malicious intent here when people are just sharing their views. |