[vidyesh]

Awesome! Feel free some or all.

Few of my queries you mostly answered elsewhere in your replies about images and how you get the data.

How did you determine which plants to list? Is it just a database of all the unique plants from all sources or just a general plant database?

I see each plant shows price + x stores, does that mean you are archiving the prices and not scraping real time? How are you determining the time interval for that?

How are you handling wrongly or typo listed plants from your sources?

Since you mentioned this is using next and django, what are you using shopify for?

Are you an affiliate for most of them or nothing like that so far?

Sorry I don't really have any valuable feedback apart from what everyone mentioned, Search is really slow.

And some or most external links have an extra / in their hyperlink, so at

  https://www.getanyplant.com/plant/4f5952a72087bce5e5c28a72c76c7563 

  https://planetdesert.com//products

Thank you.

[ryebread777]

I just list all of the plants sold at any of the stores I currently integrate with. I use a scientific taxonomic name database to make sure that the binomial names (genus, species) are legit and not miss-spelled and all of my liastings are anchored to that. I match products to binomial names through a process thats that uses a really complex regex, plus some manual labeling. Experimenting with using more ML here.

Price/availability data is updated on a nightly basis. Probably want to increase that frequency, though no one has complained about stale data yet, so not high priority.

Getting some affiliate relationships set up right now! Already have two added in the past week.

That extra / is mysterious... I will dive into that after work.

[vidyesh]

Thank you so much! I am not that experienced building large scale projects so really appreciate your replies in thing post.

I am quite surprised that shopify doesn't have hotlink protection for images!

>I match products to binomial names through a process thats that uses a really complex regex, plus some manual labeling. Experimenting with using more ML here.

Thats what I wondering. I am building something similar, region specific for books and sometimes the names are just a little off or partial or alternate names. I am currently doing a string comparison to match at least 80-90% of the words in the title, which works okay for now. So thank you for the ideas.

Your product update frequency is very interesting, I always thought scraping for price aggregation meant one has to make sure its very frequently updated. My approach is a bit different, it only scrapes on search, so not really scraping all the sites. Not the best approach, but its scary to me to scrape complete websites and that much data lol I currently am not using a db either but scraping and caching for 30mins, that specific item which now I think about is a bad idea if I want to make this a scalable project. I should start using a database indeed.

Some feedback on the UI/UX, instead of having 'All plants' selected on the homepage, it would be nice to instead have a smaller grid of plants from each type/tag on the home page itself. Selecting any of the tag would work the same as now but homepage will have more to explore because currently its just overwhelming to do anything on the homepage. I am just looking in specific tags or just searching.

Edit: This is a great resource for adding more info about pet friendly plants to the listed plants. https://www.aspca.org/pet-care/animal-poison-control/toxic-a...

[ryebread777]

Thanks for the advice! I too am surprised by that. The Django version of my site had something similar! The home page showed plants grouped into categories kind of like Netflix. But when I transitioned to react I limited the scope. But I might go back to that!

One other tip - many sites have APIs that will give you their product data. You may need to contact them about getting access. Or it may be publicly available. But that is better than scraping if it is possible.

[pastor_bob]

>Since you mentioned this is using next and django, what are you using shopify for?

Looks like they're just pointing to images hosted by others, most of which are Shopify sellers. One weird trick to save on bandwidth costs...but obviously problematic in the long term.

[ryebread777]

Currently evaluating alternatives for hosting images rn. Open to any suggestions!