[toomuchtodo]

Congrats on the launch! Huge TAM so there’s lots of room for a healthy ecosystem of competitors in no code low code. My suggestions for succeeding in this ecosystem:

* The value is in the long tail. Major services will integrate with other major services on their own. Make it easy to integrate with the smaller services.

* Enterprise features is where the rev is at, you want to be able to get into Corp customers who are willing to spend $50k-$150k/year with you because you offer automation their internal it or dev teams can’t (along with RBAC, audit logs, and the usual trimmings that an enterprise offering entails) (individual and smb personas are fine, but they are price sensitive and have higher churn)

* UX is important. Spend the resources as you scale to understand how your users are leveraging your product for their workflows; it should be magical to them. The easier it is to use, the more it’ll be used, which translates to more revenue (assuming revenue tied to tasks executed).

* Integrations will break frequently; instrument to know when this happens and to rapidly roll out fixes.

* It is crucial to be able to pause your workers as well as replay data from webhooks and polling. Also, log all the things (while redacting secrets) as data is processed. This will make troubleshooting integration issues and edge cases (which will pop up often at scale) less painful as data structures flow through your code paths.

[moabuaboud]

Are there any programmatic methods to detect integration breakages? Simply relying on unit/integration testing with mock endpoint is not effective, as it cannot capture changes made by third parties. Being notified of breakages is a solution, but not the most ideal one.

I've considered sourcing open API specs, like APIsguru.com, to scan for changes, but I was wondering if you have any other suggestions.

[toomuchtodo]

It has been almost a decade since I've been involved at such an org, so I cannot speak to the state of the art. Folks currently doing the schlep are going to be more knowledgable than myself. With that said, you are likely to be alerted in a few ways of trouble with an integration:

1. Exception handling. Whatever code you have polling API endpoints or processing inbound webhooks is likely to throw an exception if the structure of the API it's consuming or inbound webhook message formats have changed materially. I recommend handling via Sentry or a similar application error reporting mechanism for triage by your SRE or platform team.

2. API responses. There is some peril here, as every API is different. Some APIs will behave as you'd expect with respect to error codes, error messages, and request allowances, while some APIs will reply with code 200 with the error message in the body. Again, this is the value incumbents offer; they know what failure looks like for each API, and they also have a good idea of what success and health looks like at steady state. Build relationships with API partners (do you have a partner team? you eventually should) so that you have open comms with them with regards to breaking changes, and code defensively in general. Tangentially, ensure you have robust logic around deduplication of polling data.

3. User reporting. If your unit tests didn't catch something, nor did your application error mechanisms, your users will absolutely let you know if a piece of JSON element landed where it shouldn't have in a target integration.

I'd encourage you to ask around to others in this space, as their recent knowledge will be more relevant for avoiding sharp edges. Also, once you've built whatever you're building, you'll be able to (or, at least, you should if you've approached this from a systems thinking perspective and wrapped the necessary telemetry and observation tools around the machine) observe at scale what optimal and suboptimal looks like.

[molsongolden]

Are you thinking of misc. sync errors here (auth issues, temporary downtime, etc.) or API changes?

For API changes: It's not super scalable but I've used a changelog monitoring chat (Slack/Zulip/Discord) channel. Automated notifications for changelog updates for all integrated APIs. Maybe there's an assigned person for each integration and that person scans the changelog updates for their assigned services then emoji reacts to the notification messages when they have been reviewed and cleared (they create issues for required changes or confirm no changes needed).

[moabuaboud]

That's an innovative idea, I like it! Was it straightforward to set up automated notifications for the other services? Did you encounter any challenges or problems along the way?

[qwertyzxcvmnbvw]

Maybe https://www.akitasoftware.com/ could help.

[ashrafsam]

Thank you-there is huge similarity between how we think and your comment!

How do you define long tail? Fewer users?

[toomuchtodo]

Yes, long tail is smaller user base.

[ashrafsam]

Thanks!