Hacker News new | ask | show | jobs
by speedplane 2710 days ago
>I have an idea I wish to try out regarding word embeddings, and I am looking for either a minimalist or a very clearly documented library for word embeddings.

Using AI on natural language is still in its infancy. You're not going to find a clean off-the-shelf solution if you're doing something unique.

Google Cloud has a natural language AI module that makes solving certain problems pretty easy (little to no coding). The next step is to look at packages like NLTK, which is more complex but can handle a larger set of natural language processing (you need to be comfortable with Python). If neither of the above tools does the job, you're going to need to dive much deeper, and learn about the fundamentals of neural nets and natural language processing and become familiar with tools like Tensorflow.

Natural language processing is a much more open field than image processing, despite a seemingly much lower bandwidth and data size. As far as data-size, all of Shakespeare's works can fit into a single iPhone image, but understanding what's going on in natural language is often a far more difficult task than what's going on in an image.

1 comments

I apologize for my late edit, and thank you for your response. I just need a minimal codebase that generates a word embedding that satisfies these quasi linear relationships.