I would encourage more development be done to have the type cover more of the front end code. Your development environment must not be of much help due to the heavy use of the "any" type throughout the codebase, which essentially escapes out of the type system.
Thanks, it started out as regular, valid JS, then I figured I might as well convert it to TypeScript, and I did so (perhaps too) quickly. I do plan to add stronger typing later.
What advantages did you get from typescript? Seems like type safety is a huge trade off from what you lost by adding those amounts of complexity, especially when you’re not really utilising it.
I see that you are scraping the results directly, curious if you’d tried out Google’s json search api? Any thoughts about it? I was about to need it or similar for a project.
Google's json search api gives you a consistent easy to parse api.
However it requires a Google API key, which limits free usage.
Client-side scraping scales effectively, as it distributes the reads onto all the clients. But it's also more brittle, as Google could change a small implementation of their search and break all the scraping functions.
I've tried it with the C# wrapper library, so using plain old methods and C# objects rather than touching the JSON directly.
It's not too bad, but not free as far as I'm aware and the result snippets they give you are quite limited in length. Also I found it a little tricky to get good results on some queries when I tried it out.
OP is being bit dismissive, but IMHO the key feature of a metasearch engine is the processing of results, not the interface. Filtering Google’s results is technically a metasearch engine... but it’s a fairly trivial example.
I was looking into how google was implemented initially with the pagerank algorithm and I think I am getting closer. Do they still use the same algorithm? I ahve no concept of how things behave when they scale or how to scale for that matter.
I think they just use BrainRank for search now. It still incorporates the basic premise of pagerank, but the rules are not hand written anymore, it's all done with Tensorflow.
I would encourage more development be done to have the type cover more of the front end code. Your development environment must not be of much help due to the heavy use of the "any" type throughout the codebase, which essentially escapes out of the type system.