Why is it so hard?
- Variability: A single meaning can be expressed in multiple different ways.
- Ambiguity: A single expression can have multiple meanings depending on the context (“Go to the bank” - 😉).
- Sparsity: Basically, Zipf’s Law.
- Robustness(?)
- Context Dependence
- Language Diversity
- In other words, uncertainty. That’s why, ML is absolutely necessary for NLP.