[smickie]

I have repeatedly had clients on a budget choose Magento over Shopify due to a lack of fine grain control.

It's very frustrating that they can release a POS system, however fail at, for example, complex sales reports.

I wish they'd get it together because they would blow everyone out the water.

[orenmazor]

I work on the new Shopify Reports product and we would love to hear more about what kind of complex sales reports you're looking for.

hit me up here, on twitter (@orenmazor), or email: [e-mail]

[jacques_chester]

Suggestion: provide a dimensionally-modelled reporting schema, it will be much easier to query.

Kimball's The Data Warehouse Toolkit is the definitive tutorial on the subject.

[orenmazor]

Our first sprint on this product was following the first principles as outlined by that book.

We found that the strategy works really well on paper but there are significant scaling issues on the querying side when you throw massive amounts of data at it. A simple sales report over cities for one of our larger shops under moderate load would could take 5-10 seconds to generate, which is pretty unacceptable. Caching would only take us so far because of how much data gets ETL'd every moment.

I definitely don't discount the dimensionally modelled strategy, but to make it proper fast, and not 1990's let-me-hit-report-and-go-get-a-coffee fast, you might need to write your own OLAP stack that's optimized for what you need[0]. I'd also do it in go or c.

Once we ship, we'll do a technical post on what worked and what didn't.

[0]I'd love to be proven wrong on this, so if you can generate fast reports with massive amounts of data ETL'd in real time, I'd love to hear from you. [e-mail]

[saurik]

I fail to understand why waiting five seconds to generate a report that will impact my business decisions is "unacceptable". I have numerous reports I've implemented that take over a minute to generate; they are complex queries that sometimes involve interesting statistics, and the idea of calling them outdated solely because they don't return data in a fraction of a second seems like a fundamental misunderstanding of what businesses do with reports.

[jacques_chester]

How massive are we talking? What's your finest grained fact table? Are we partitioning fact tables by customer? What's your current underlying store?

You're actually at the coalface and I'm the bookworm, so I am totally prepared to be schooled on this one.

[orenmazor]

We started out with mondrian+mysql, but quickly had to drop mysql ("mysql is great until you want to put data into it and then get it back out again" - unattributed, to protect the guilty). Our primary work was with postgresql in the beginning. which, in my opinion, is a pretty solid database to work with overall.

We did partition all of our dimension and facts tables, which helped a great deal. Aggregates were a problem that is specific to us, so we couldn't cheat the usual way that reporting servers do.

The other problem is that our stack is suddenly fully of things like java and olap and postgresql, which made onboarding people who wanted to help, and just debugging, a pain.

I like that comment about the coalface/bookworm, but sometimes it takes somebody on the outside to see what I'm missing.

[beagle3]

With all due respect to pgsql, mondiran and mysql - you're using the wrong toolkit. You need a column store, like Vertica, kdb+, TimesTen (now Oracle?), Sybase anytime (I think that was what it was called).

You can do very well at much, much lower cost in the Python world: pandas, PyTables, or even just straight numpy.

Seriously, using any of these would make the report generation time basically zero, and you'd just have to make your ETL work quickly enough to feed it; how well this can be solved depends on how you store the original data ("pre-facts").

The book written by this guy http://blog.wesmckinney.com/ (and the guy himself, if you can get him) will probably advance you way more than experiments.

[adammcnamara]

A beta of Shopify Reports actually launched today as well. We're starting with point of sale customers, then bringing all merchants on shortly. http://docs.shopify.com/manual/your-store/8-reports

[fourspace]

Super excited to see what these can do. Wish they were available for Pro plans as well, as I'm building similar reports for our store as we speak.

[theory11]

This is fantastic.

[HyprMusic]

Agreed. The control they give their theme developers is shamefully woeful. You're actively encouraged to use Javascript as a work-around for their very limited templating sytem. If they were to add querying, or even just a few more options for selecting products (e.g. by type, brand, etc.) then Shopify would be a great contender to other e-commerce stores. Currently it feels they're only focused on things that can bring them more revenue.

[ericingram]

I'm the founder of Forward Commerce (http://getfwd.com), a new platform focused on giving developers more template control without all the pain.

One of our early adopters was a very notable/early Shopify customer who encountered the kind of problems you describe with templates and customization. I am pleased to say they have been successful using Forward and Shopify combined in order to solve these problems, and are planning to migrate completely when the time is right.

Shopify is a great solution for small business without irregular catalog and checkout requirements. I am however surprised that they haven't done more to support businesses that grow beyond this stage.

[HyprMusic]

Your project looks interesting, I love the fact it's open source with premium hosting - the ideal way to do it if you ask me.

[brockf]

>> Currently it feels they're only focused on things that can bring them more revenue.

That makes a whole lot of sense to me, and would be the only acceptable path to take in the eyes of their shareholders.

[ogig]

I felt like that too. My country payment options are lacking, api has some old bugs and shortcomings like no SKU search, taxes system doesn't fit for many business that have products with different tax type, no multilang, and plenty of ugly liquid and js solutions.

Also, checkout has a small legal nitpick here in europe, where a check has to be somewhere and in shopify is not. That means plenty of EU stores might be in a grey zone. Same with cookies in the checkout, but that's another story. I guess they will fix it but i guessed that 7 months ago.

I like Shopify in many ways, but after two (small) stores with them i have two main issues:

- EU distant. I feel US market is the main thing for them and EU is left behind using weird solutions for complex tax cases or pricing, few payment gateways, and so on. The checkout issue ilustrates this.

- Pushin new features while not fixing old ones. Shopify 2, Shopify Payments US, and now the POS are the examples. Meanwhile my app has to do crazy stuff to sync my products ids with their native ids.

Said this i will probably keep using it but Prestabox is tempting me for the next project. Any experiences with it?

Also, Shopify guys, the 15 days in advance warning for an api change in mid august was nice!

[nahname]

I agree, in some areas they are simply amazing. In others, they have a lot of work to do.

For example, the devs at my company that have had to use shopify hate it. Their systems are based on hacking javascript for everything. The deployments are very painful too. No way to see what it will look like without previewing it live on the site.

[rquantz]

Agree that testing/deployment is a pain. The solution I've come up with is to use the shopify theme manager app, which syncs local files to the online store, then have to two copies of the repo on my local machine, corresponding to the live store and a testing store. It's a pain, but it's Bette than hacking on the live theme files.

[erichurkman]

Their shipping options completely suck for anything but trivial sales, too, unless it has changed in the last month or two.

(Example: most products are free shipping but a few large products are not free; there was no control to exempt products from free shipping, nor any control to say products were not eligible for certain shipping methods. Not all merchants can overnight a 50kg box, for example.)

[gravitronic]

Hi, you can set up weight-based shipping rates so products over a certain weight have different shipping rates applied. You just need to tell Shopify the weight of those products. http://docs.shopify.com/manual/settings/4-shipping/4-shippin...

If that's not good enough there's apps like Better Shipping that can set up more complex shipping rules - http://apps.shopify.com/better-shipping

[AYBABTME]

Have you ever thought of using FBA?

  http://services.amazon.com/content/fulfillment-by-amazon.htm

You can opt-in their Prime program and offer free 2-days shipping to your customers.

I think there was FBA integration in Shopify but they're dropping it, but an app still let you integrate with FBA.

[erichurkman]

Yeah, I tried to push for that. The margin on the products was too small. Due to some nasty wording in franchise agreements, the actual markup was 25% at most, terrible rates for retail. There just wasn't enough room to play with discounts/promotions as well as external servicing. At least not at their low volumes. I've seen other FBA projects go quite well though.

[tvladeck]

"Fail" implies it's a priority. Maybe they think this is a bigger problem for their customers?

[solaris152000]

You find the Magento reporting much better? I've never tried reporting in Shopify, but Magento out of the box is very very limited.

[smickie]

Magento does everything if you wade through the data. Like if you wanted stats on... off the top of my head, abandoned shopping carts by region and product, you could.

I think in Shopify you can view how many abandoned carts there are in the orders section, and it's not filterable.

I mean how difficult would be for Shopify devs to add a reports section on stuff like this. Just tabulated data would do, I don't need any animated graphs. It's very frustrating because the data is just out of reach in Shopify.

Edit: I'm not a Magento advocate here, I really really want Shopify to get better quick so I can stop using Magento.

[jmccree]

Reporting can be surprisingly complicated. You can allow users to run raw SQL queries, or have a reporting interface that is basically is an SQL query builder behind the scenes, and have extremely powerful reporting abilities. The problem is you might have a user create a query that takes 5 minutes to run, and then refresh the page 15 times killing your db server/s. Many places have a separate DB read replica (or multiple replicas) just to allow this type of reporting run by certain users.

For an environment like shopify (which I've never used, so speculating), their reports are much more likely optimized database queries or views, or even created from specially stored statistics data not from querying raw db tables, with caching or pre-generation at some point. This means there's a lot of thought behind every report, and what's possible with the current db/stats schema. Adding a column to a report might mean modifying the db schema.

The final issue is that user interface complexity goes up as reporting flexibility goes up. A super powerful interface that few people take the time to use is worth less than a limited interface that provides value to more people. The task is attempting to satisfy power users and more complex needs, while providing a very useful default functionality out of the box.

[jacques_chester]

This sounds like a case where providing users with a dimensionally-modelled reporting schema would be a big win. It would help both with the slow query problem and the super-complicated interface problem.

[orenmazor]

I appreciate the difficulty of trying to run a business without this information being available to you.

Once you start juggling the vast amount of data that needs to be stored, fetched, calculated and referenced to provide a report, it becomes a lot more than "why dont you just". And thats before scaling the system to the non-fail-whale standard that we run our infrastructure at.

But it is coming very soon. In fact, it's already available to PoS customers.

As I mentioned, feel free to ping me to talk more about this. I would love to have more input.

[snookca]

Specific to your abandoned carts comment, the view is filterable. Type in "Canada", for example, and it'll show all abandoned carts in Canada. We'd love to learn more about what you're looking to accomplish that is difficult to do right now.

[solaris152000]

What if you wanted sales by category?