[chousuke]

Steampipe is unbeliveably powerful.

Writing custom scripts for even the simplest of queries comes nowhere near the convenience of using PostgreSQL to get what you want. If you wanted to find and report all instances in an AWS account across all regions, with steampipe it's just:

   SELECT * FROM aws_ec2_instance;

Even the simplest implementation with a Python script won't be nearly as nice, and once you start combining tables and doing more complicated queries, there's just no competition.

[Hikikomori]

Tried steampipe out to replace my simple python script. It wasn't able to handle our 1k+ accounts and multiple regions, initialization took over 1 hour before I was even able to do a query.

Meanwhile my python script can run any API calls in all our accounts and regions and finish in a few minutes. Maybe 100 lines of python using boto3 and multiprocessing that outputs to json or yaml.

[nathanwallace]

Python and boto3 are very powerful in many cases for sure - glad they work well for you. No doubt that multi-region for 1k+ accounts is a large surface area, so we'd love to learn from your experience! Performance is an ongoing focus area for us, including radical improvements in memory usage and startup time in our most recent version - https://steampipe.io/blog/release-0-16-0

[xwowsersx]

1K accounts? What're you doing over there exactly? I'm intrigued!

[Hikikomori]

Long story but currently it's a mix of per team or per system accounts to have clean permission borders (so one team cannot touch another teams resources). Though we also support account multi tenancy through scoped actions (tag or path based).

[phillu]

I don't agree that this simple query wouldn't be as nice with boto3. But combining different services is definitely a nice feature, if you have the use case for it.

[chousuke]

the fun part of that query is that it'll search across all regions that you have configured steampipe to use. If you did that in Python, you at the very least have to manually iterate over each region, handle pagination yourself and combine the results.

It gets much more convenient when you want to ask more complicated questions like "What is the most used instance type per region" or "How much EBS capacity do instances use across all regions, grouped by environment tag and disk type?"