[donatj]

I was telling a friend recently about how there was this "golden age" when you could access all sorts of free APIs, and how I still long for this time.

I remember the public Netflix API, Twitter APIs and Flickr API with particular fondness. My personal site was a big mashup of all of my data.

I also abused the hell out of Yahoo Pipes - I would run RSS feeds through like 15 different languages with Babelfish before back to English, just for kicks.

My friend seemed very skeptical such a time ever existed.

[duxup]

It reminds me distantly of an earlier time when just the web was sort of owned ... by the web folk.

In the sense that even corporate sites if you found it would have a little corner where the 'webmaster' had a page that mentioned the server, or his cat, or some silly pic. Some sort of character or tidbit before any of the branding drones were really aware of the web. All just because the 'webmaster' was the only one really in charge / who understood the site was even there and they wanted to share.

I suspect to some extent the APIs were the same. Someone who really didn't mind was all "Yeah sure if someone wants to see what I did.. awesome."

[asudosandwich]

>> a little corner where the 'webmaster' had a page that mentioned the server, or his cat, or some silly pic.

Or a “links” page. I haven’t seen that in a long while. No affiliate garbage or anything just a page linking to other sites that the webmaster liked or whatever. It’s hard to remember when that fell out of fashion but it did seem to add a personal touch as well.

[mattl]

I used to beg people I met online to host pages for me, give me shell access, etc. I was a child in the very early 90s with basically no money.

[wastedhours]

The same, in fact my life would be very different if a random guy from (I think Kansas, given I'm from the UK) on a forum bought me a domain and gave me a slice of his dedicated hosting when I was mid-teens.

I've always wanted to pay it forward in the same way, but lots of things on the web seem overly complicated now that'd make that hard to do, and I've lost attachment to most communities.

[mauricedenassau]

It's not very common, but I had people begging my friends for this kind of services.

[flingo]

raspberrry pi in home-network DMZ, with fail2ban, unattended-upgrades, and a free dynamic DNS service gets you most of this. (plus, you're root, so you can run things more complicated than web-services)

[wlll]

> It reminds me distantly of an earlier time when just the web was sort of owned ... by the web folk.

I remember those times, I miss them. I had a modem back in the late 1990s and used to buy .net magazine (in the UK, back before there was a framework of the same name) on my way home from school and it had the number of people estimated to be on the Internet printed on the spine. It all seemed too good to be true, we were worried it might get shut down by governments. There was the TV program "the net" (https://en.wikipedia.org/wiki/The_Net_(British_TV_series)) that used to give you an "info dump" at the end that you were meant to record on your VCR and play back frame by frame.

In Jan 2000 I got my first proper job at a web hosting company and used to read Wired magazine before it became (as far as I recall) fascinated by the stock market.

I miss the optimism and simplicity of those times.

[streb-lo]

> I miss the optimism and simplicity of those times.

It seems like every generation (minus those who came of age in WWI/II) think this is true for them.

Is it really true, or are we just all reflecting back to when we were younger and the world was simpler because we understood less.

[wlll]

To be clear I was speaking specifically about the Internet/Web and not the world in general. I would say that there was a lot of optimism for the web back then, and it was definitely simpler.

[Causality1]

Back when I had three hundred bookmarks instead of, like, six. God I'd kill to still have that list.

[time0ut]

I remember these days fondly. Surfing the web on 486 at 28.8kbps. it felt slow, but worth it. Most information to be found existed because someone thought it was worth sharing. A golden age to be sure.

[GrumpyNl]

Thats the web we love.

[diggan]

> I also abused the hell out of Yahoo Pipes - I would run RSS feeds through like 15 different languages with Babelfish before back to English, just for kicks.

Yahoo Pipes was one of the greatest services I used, just when I started getting into programming. Maybe it was so cool because I was naive, but I really miss being able to pipe services together in the same way. Anyone know of any similar attempts that is open source + offers a hosted version with paid plans?

[onli]

Not sure about open source and paid hosting, but there is node-red, https://nodered.org/, which is open source and easy to get started with. Combining stuff is something Zapier excels in, https://zapier.com/, and the free tier can suffice for some tasks (not open source). You could also try my attempt at a spiritual pipes successor, https://www.pipes.digital/ (but it's also not open source). If there is something missing there to reproduce how you used Yahoo Pipes I'd definitely be interested in hearing from you, so I can restore it :)

[diggan]

So, the requirements of open-source + have entity with paid hosting are both equally important. First one to ensure I can continue using whatever I setup on the hosting, in case the entity behind it cannot. And the paid hosting is important because it gives better odds towards the service actually sticking around.

What I likes with Yahoo Pipes compared to NodeRED (at least as far as I looked at NodeRED, I might be wrong) is that Yahoo Pipes worked out-of-the-box with services out of the box. I seem to remember that you could use the Google Search API for example, with Yahoo Pipes and pipe that into other things. That's what Zapier does as well, but with less flexibility than NodeRED.

So I guess my dream would be something like the integrations provided by Zapier but with the UI and flexibility of NodeRED.

Haven't seen pipes.digital before, I'll take a look as it looks interesting, but for anything serious, open source is a hard requirement (gotta learn from the Yahoo Pipes history :) )

[r-w]

I don’t think those requirements will be met until people like us who miss it put our money where our mouth is ;)

[diggan]

I think people like us (at least me) miss a lot of things, some more important than others, and we have to carefully choose what we spend out time on :)

[onli]

I'm about to open source it now. Which means I just did, but haven't announced it yet. https://github.com/pipes-digital/pipes now contains a very new version you could run locally.

r-w is right though: More users paying for pipes would allow me to invest more time into it and have more requirements covered.

[routerl]

I haven't used it but https://n8n.io/

It reminds me of zapier, which reminds me of Yahoo pipes.

[ivanfon]

From a quick search, it looks like someone is recreating Yahoo Pipes here: https://www.pipes.digital/

Unfortunately, it looks like it's not open-source, their Github repo is only for bug reports: https://github.com/pipes-digital/pipes

[egfx]

Yeah as I mentioned https://news.ycombinator.com/item?id=21641615

A Yahoo pipes! clone really needs some open source love. Too bad the maintainer is more interested in making it a one man SaaS.

The video is interesting because I thought he was talking specifically to me about the http://2fb.me product. But anyway I think this is a reminder that even though API’s have gotten more restrictive that more creative ways of using them bubble to the surface. :)

[onli]

> A Yahoo pipes! clone really needs some open source love. Too bad the maintainer is more interested in making it a one man SaaS.

I guess that's fair.

I want(ed) to run Pipes as an SaaS because I think that model would be highly advantageous to me, and that would be very advantageous for Pipes. But that with the predecessors history as background this approach is a critical concern is something I do understand. I will now try to have my cake and eat it too: https://github.com/pipes-digital/pipes contains a new FOSS version that is meant to exist in parallel with pipes.digital, as long as that model works out. So worst case now is this FOSS approach fails, and then only the SaaS would die, not the software.

[reubano]

Check out my project, riko. https://github.com/nerevu/riko Open source, but no hosted plan.

[erikig]

For a self hosted Yahoo Pipes alternative I’d recommend checking out Huginn - it’s been featured on HN a couple of times and the creator is tectonic@HN

https://github.com/huginn/huginn

[zwily]

I don’t, but Node Red is giving me the same fun feeling I had with Yahoo Pipes, but for Home Automation.

[hundchenkatze]

Dark is similar. I think they're still in private beta, but I got my invite pretty quickly.

https://darklang.github.io/docs/introduction

[akavel]

I believe https://luna-lang.org has plans like this, though they're kinda definitely not hurrying things...

[anonytrary]

I remember when you could just stream in every tweet and reddit comment ever posted with a few lines of code. There used to be dedicated API methods for doing so. Now, there's all sorts of namespacing, rate limiting, pagination, and upper bounds on data access that make this impossible, or at least infeasible.

[sneak]

Twitter wants to capitalize on the fact that tweets are massively public, and they also want to capitalize on the fact that they are not.

Researchers are passing around lists of tweet IDs ("dehydrated", they call them) that can be "rehydrated" (that is, turned back into full tweets) if you have the right permission from twitter to do so.

The whole setup is really shameful.

It would be de-facto illegal to build a "Google for Twitter" today. I settled on doing it for ActivityPub/Mastodon because it's less likely I'll get sued into oblivion for creating a search engine that way.

[snisarenko]

Hi sneak,

I am working on some fediverse aggregator projects. Let me know if you want to talk.

https://mastodonia.club

https://pixelfed.club

[sneak]

You have no standard contact methods available on any of your sites that do not require registration. Consider publishing your email address; I am happy to chat. My email is in my profile and on my website.

[snisarenko]

Sent you an email

[banana_giraffe]

This reminds me of an endpoint that Google used to run. It was a never ending stream of RSS feeds. Anyone could subscribe (it was just a never ending HTTP GET, if I remember right), and be told which blogs had updated in near real time.

Nowadays you'd need to sign up for an API key, probably pay some amount of money, and provide twenty different forms of contact to use something like that. That's assuming it was allowed to exist in the first place, since it might take views away from google.com.

[Lammy]

> I also abused the hell out of Yahoo Pipes - I would run RSS feeds through like 15 different languages with Babelfish before back to English, just for kicks.

For a while it was common to find SEO-spam sites composed entirely of posts generated this way. They would translate from a source language back to it in a roundabout way and end up with an article that was "different enough" to count as unique content to Google.

[smcl]

A British MP (I cannot for the life of me remember who) came under fire for owning a company made its money doing this - take existing content, shuffle the language round a bit and publish it with loads of ads.

[matthewheath]

Likely to be Grant Shapps

> In 2012, Google blacklisted 19 of the Shapps's business websites for violating rules on copyright infringement related to the web scraping-based TrafficPayMaster software sold by them

https://en.wikipedia.org/wiki/Grant_Shapps#Business_ventures

[smcl]

Yes! That's him. I only knew about that one, but this other one mentioned afterwards is hella shady too:

> It cost $497 and promised customers earnings of $20,000 in 20 days. Upon purchase,

> the "toolkit" was revealed to be an ebook, advising the user to create their own

> toolkit and recruit 100 "Joint Venture Partners" to resell it for a share of the

> profits

Sort of like a pyramid scheme, really. Incredibly this appears to have scarcely affected him, he is still an MP :-O

[pevezzac]

I also remember vividly those optimistic times. You should show this video to your friend as proof: https://www.youtube.com/watch?v=6gmP4nk0EOE

[giantrobot]

There was a lot of interesting promise in Web 2.0 that was completely wasted and then died. I think a big part of the problem was even things like Yahoo Pipes were too complicated for a lot of people and anyone trying to use third party APIs for commercial purposes ran afoul of EULAs or just plain old rent seeking. Once privacy invading advertising became the norm APIs were further restricted or discontinued because the user wasn't running a bunch of client side code tracking and scraping all their behavior on a web page.

It didn't help that a lot of Web 2.0 darlings sold out to on-the-way-out Web 1.0 companies \cough\Yahoo\cough\. Yahoo's management couldn't even monetize money, let alone Web 2.0 properties. So instead we got social media silos. You can put stuff in but good luck ever getting it out. You can share it with whomever so long as they also join the same silo.

[cjhopman]

There are 3 big problems with open APIs

1. they enable people to do things that other people think shouldn't be done

2. people get upset at companies when (1) happens

3. people get upset at companies for removing or restricting apis when, or in fear of, both (1) and (2).

[cortesoft]

I think you are missing what is probably the biggest factor: they can be very expensive to run if successful.

If you create a popular API, people are going to find creative uses for it, and because they can, by definition, be automated, you can get rapid growth in traffic with not that many users.

There is a bit of a 'tragedy of the commons' that goes on, because the people writing apps that consume the API have no incentive to moderate their usage, or try to be efficient.

Since the company that is providing this API is paying for the resources to run it, they can quickly get very expensive. Unless there is a clear financial benefit for allowing it to continue, most companies will shut them down eventually.

[Nextgrid]

Is it actually expensive, or is it just a result of everyone moving to cloud providers that charge an arm and a leg for performance equivalent to a cheap laptop (when you account for "CPU credits" and all that) and try to nickel & dime you on everything, including bandwidth (despite bare-metal providers somehow being able to stay in business by offering unmetered 1Gbps bandwidth)?

[johannes1234321]

Also the API forms a committed interface you are stuck with. Whatever way you want to "improve" your offering you have to make sure it's compatible with the old API, at least for a while. Especially as you can't reach your consumers.

[rplst8]

Couldn't this all be solved with a p2p network that hosts restful services?

[ericflo]

The framing of open APIs disappearing because of bad actors doesn't ring true to me. In my view, the golden age of APIs disappeared one-by-one as tech companies realized they won their respective markets and consolidated their power.

[bryanrasmussen]

I think what you said, yes, but also bad actors. Basically they disappeared because what business purpose did they serve, they were a cost center not a profit center - bad actors just increase the cost.

[marcocampana]

Those times were absolutely great. It's only human to look at the past with a feeling of melancholy and forget how great is the present and the long way we came. Today you can actually build your own APIs with very little knowledge and effort: easy to get started frameworks, free or inexpensive hosting in the cloud, lots of freely available data. That's pretty awesome too.

[amatecha]

Heck yeah! Years ago I had a nice little feed of my latest Tweets and Flickr photos on my personal website! It was fun incorporating them cleanly/seamlessly into my custom design. One day I'll get my site going again and have fun stuff like that -- if the 3rd party APIs even allow it these days :P hehe

[redwall_hp]

Even Google Search had this. I read an old O'Reilly book back then that was centered around API mashup projects (remember that term?) using the many Google APIs, which have mostly been shut down.

[joshspankit]

It surprises me that we don’t have crypto-based API micro-transactions yet.

Even if you had to pay 1 satoshi for 10 queries, it would go a long way towards making APIs viable long-term.

[adrianN]

What's the transaction rate for bitcoin these days? Is it still a handful of transactions per hour? What are the transaction costs?

[ric2b]

There's a layer 2 technology called Lightning Network that lets you transfer Bitcoin almost for free and in seconds, no transaction number limit. The biggest issue is liquidity, it's not easy to move large sums in a single transaction, but that shouldn't matter much for micro-payments.

There's even ongoing work on making paid API's work seamlessly by adding some middleware, using the HTTP 402 (payment required) status code, no usernames/e-mails/passwords involved: https://lightning.engineering/posts/2020-03-30-lsat/

(And yes, LN is open source, non-custodial and moves real Bitcoin, not IOU's or other tokens. It's actually an open spec with at least 3 major implementations in 3 different languages.)